2024-09-19T03:46:18.336 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T03:46:18.361 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T03:46:18.451 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911619' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIDQp+BKaM0CTtwtUWutINilK4KvQEjSfuBRpCNTQarA/1e5NUaWGxDsJoah26t+o7hl/wa/NH3MdOfZHVv5sOc= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF4jVpYhhLAsV1b5JkZ7VVhmZ2Ut7rSrdg8VXM90nuhZY6dbcr8ESTybVO6Gf7zAj+DYewd0qhaQQ83IS3Nmotk= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOUQrknR6lCGdDuSUu5GT5V9u3lHKEZ+effXva0D7IxPguXhTKDpuKja5PP8lNLY0AQ8pq8/9lqeT29Irsu46ws= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T03:46:18.451 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T03:46:18.452 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T03:46:18.452 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T03:46:18.454 INFO:teuthology.task.internal:Checking packages... 2024-09-19T03:46:18.475 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T03:46:18.476 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T03:46:18.476 INFO:teuthology.packaging:ref: None 2024-09-19T03:46:18.476 INFO:teuthology.packaging:tag: None 2024-09-19T03:46:18.476 INFO:teuthology.packaging:branch: squid 2024-09-19T03:46:18.476 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:46:18.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T03:46:18.709 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T03:46:18.711 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T03:46:18.717 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T03:46:18.717 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T03:46:18.729 INFO:teuthology.task.internal:Saving configuration 2024-09-19T03:46:18.749 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T03:46:18.755 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T03:46:18.780 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:40:56.582734', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIDQp+BKaM0CTtwtUWutINilK4KvQEjSfuBRpCNTQarA/1e5NUaWGxDsJoah26t+o7hl/wa/NH3MdOfZHVv5sOc='} 2024-09-19T03:46:18.800 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:40:56.584227', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF4jVpYhhLAsV1b5JkZ7VVhmZ2Ut7rSrdg8VXM90nuhZY6dbcr8ESTybVO6Gf7zAj+DYewd0qhaQQ83IS3Nmotk='} 2024-09-19T03:46:18.823 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:40:56.581050', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOUQrknR6lCGdDuSUu5GT5V9u3lHKEZ+effXva0D7IxPguXhTKDpuKja5PP8lNLY0AQ8pq8/9lqeT29Irsu46ws='} 2024-09-19T03:46:18.823 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T03:46:18.829 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T03:46:18.829 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T03:46:18.829 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T03:46:18.829 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T03:46:18.916 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffa0494bbe0>, signals=[15]) 2024-09-19T03:46:18.917 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T03:46:18.922 INFO:teuthology.task.internal:Opening connections... 2024-09-19T03:46:18.922 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T03:46:18.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:46:19.006 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-09-19T03:46:19.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:46:19.084 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-09-19T03:46:19.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:46:19.162 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T03:46:19.168 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-09-19T03:46:19.187 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-09-19T03:46:19.187 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-09-19T03:46:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-09-19T03:46:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-09-19T03:46:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-09-19T03:46:19.245 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:46:19.245 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-09-19T03:46:19.245 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:46:19.246 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:46:19.246 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-09-19T03:46:19.246 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:46:19.246 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:46:19.247 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:46:19.247 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:46:19.247 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:46:19.247 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:46:19.247 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-09-19T03:46:19.269 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-09-19T03:46:19.288 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-09-19T03:46:19.289 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-09-19T03:46:19.347 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-09-19T03:46:19.347 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-09-19T03:46:19.347 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-09-19T03:46:19.347 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:46:19.347 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-09-19T03:46:19.347 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:46:19.348 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:46:19.348 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-09-19T03:46:19.348 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:46:19.348 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:46:19.348 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:46:19.348 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:46:19.348 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:46:19.348 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:46:19.348 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-09-19T03:46:19.370 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-09-19T03:46:19.388 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-09-19T03:46:19.388 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-09-19T03:46:19.446 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T03:46:19.447 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-09-19T03:46:19.447 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T03:46:19.447 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T03:46:19.447 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T03:46:19.447 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-09-19T03:46:19.469 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T03:46:19.479 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T03:46:19.484 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T03:46:19.484 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:46:19.487 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:46:19.489 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:46:19.506 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T03:46:19.512 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T03:46:19.512 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:46:19.544 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:46:19.546 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:46:19.583 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T03:46:19.655 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T03:46:19.669 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T03:46:19.670 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T03:46:19.670 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T03:46:19.670 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:46:19.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:46:19.688 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-09-19T03:46:19.745 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-503.el9.x86_64 2024-09-19T03:46:19.746 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-503.el9.x86_64 2024-09-19T03:46:19.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-09-19T03:46:24.550 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.3 MB 00:03 2024-09-19T03:46:26.434 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-19T03:46:30.993 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-19T03:46:32.477 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-19T03:46:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 23 MB 00:04 2024-09-19T03:46:43.032 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T03:46:45.005 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T03:46:45.005 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T03:46:45.097 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T03:46:45.113 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:46:45.114 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-19T03:46:45.114 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-19T03:46:45.114 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2024-09-19T03:46:45.114 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:46:45.114 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 79 M 2024-09-19T03:46:45.114 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 126 M 2024-09-19T03:46:45.115 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-19T03:46:45.250 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 312 kB/s | 36 kB 00:00 2024-09-19T03:46:46.134 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-09-19T03:46:46.784 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-19T03:46:47.392 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-19T03:46:47.393 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:46:47.393 INFO:teuthology.orchestra.run.smithi019.stdout:Total 35 MB/s | 79 MB 00:02 2024-09-19T03:46:48.100 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-19T03:46:48.485 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-19T03:46:48.486 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-19T03:46:49.475 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-19T03:46:49.475 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-19T03:46:52.031 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-19T03:46:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T03:46:52.760 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:46:55.510 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:46:55.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:47:00.854 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:47:00.923 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:47:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:47:45.114 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:47:45.155 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:47:45.504 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:47:45.504 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T03:47:45.504 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T03:47:45.504 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T03:47:46.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T03:47:46.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:47:46.300 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-19T03:47:46.300 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T03:47:46.301 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T03:47:46.301 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T03:47:46.301 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T03:47:46.301 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:47:46.301 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-19T03:47:46.686 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T03:47:46.778 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T03:47:46.778 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T03:47:46.779 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T03:47:46.779 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T03:47:46.779 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T03:47:46.779 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T03:47:46.779 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T03:47:46.779 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T03:47:46.837 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T03:47:46.838 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T03:47:46.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 82.el9 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8274670 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-09-19T03:47:46.873 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T03:47:46.874 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-09-19T03:47:46.875 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:47:46.875 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T03:47:46.875 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T03:47:46.876 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 5.14.0-508.el9.x86_64 2024-09-19T03:47:46.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T03:47:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-09-19T03:47:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T03:47:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-09-19T03:47:48.944 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-09-19T03:47:48.960 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.RJhZ3t1xhC 2024-09-19T03:47:48.960 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RJhZ3t1xhC 2024-09-19T03:47:49.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.RJhZ3t1xhC 2024-09-19T03:47:49.171 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.RJhZ3t1xhC is 6KB 2024-09-19T03:47:49.220 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.RJhZ3t1xhC 2024-09-19T03:47:49.235 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T03:47:49.303 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T03:47:49.303 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T03:47:49.303 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T03:47:49.304 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T03:47:49.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T03:47:49.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-09-19T03:48:19.461 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T03:48:19.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-19T03:48:19.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:49:19.465 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T03:49:28.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-19T03:49:28.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:49:31.964 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-09-19T03:49:32.369 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-19T03:49:32.369 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T03:49:32.369 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-09-19T03:49:32.384 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-508.el9.x86_64 2024-09-19T03:49:32.384 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T03:49:32.385 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T03:49:32.385 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T03:49:33.386 DEBUG:teuthology.parallel:result is None 2024-09-19T03:49:33.387 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T03:49:33.394 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T03:49:33.395 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T03:49:33.397 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T03:49:33.400 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T03:49:33.419 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T03:49:33.425 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T03:49:33.432 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T03:49:33.432 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T03:49:33.453 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T03:49:33.460 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T03:49:33.589 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T03:49:33.596 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T03:49:33.596 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:49:33.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:49:33.614 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:49:33.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:49:33.631 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T03:49:33.633 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T03:49:33.656 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T03:49:33.707 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:49:33.712 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:49:33.716 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:49:33.735 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:49:33.742 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:49:33.753 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T03:49:33.755 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T03:49:33.762 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T03:49:33.762 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T03:49:33.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T03:49:33.768 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T03:49:33.796 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T03:49:33.805 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T03:49:33.805 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T03:49:33.831 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T03:49:33.836 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T03:49:33.852 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:49:33.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:49:33.982 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:49:34.061 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:49:34.106 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:49:34.106 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T03:49:34.172 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:49:34.218 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:49:34.273 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:49:34.356 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:49:34.399 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:49:34.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T03:49:34.465 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:49:35.647 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T03:49:35.829 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:49:35.890 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T03:49:35.918 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:49:35.919 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T03:49:35.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-09-19T03:49:35.989 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-09-19T03:49:35.992 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-09-19T03:49:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T03:49:36.068 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T03:49:36.093 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T03:49:36.498 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T03:49:36.506 INFO:teuthology.task.internal:Starting timer... 2024-09-19T03:49:36.506 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T03:49:36.558 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T03:49:36.651 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T03:49:36.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-09-19T03:49:36.723 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-09-19T03:49:36.725 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-09-19T03:49:36.807 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-09-19T03:49:36.809 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-09-19T03:49:36.871 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-09-19T03:49:36.873 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T03:49:36.874 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-09-19T03:49:36.904 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-09-19T03:49:36.904 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-09-19T03:49:36.929 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-09-19T03:49:36.929 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-09-19T03:49:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-09-19T03:49:36.955 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-09-19T03:49:36.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T03:49:36.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:49:36.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T03:49:37.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:49:37.013 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T03:49:37.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:49:37.042 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T03:49:37.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-09-19T03:49:37.087 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-09-19T03:49:37.159 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-09-19T03:49:37.196 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T03:49:37.206 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T03:49:37.209 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T03:49:37.323 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T03:49:37.333 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T03:49:37.334 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi111.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T03:53:43.053 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-09-19T03:53:43.055 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-19T03:53:43.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:53:43.135 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-09-19T03:53:43.213 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-19T03:53:43.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-19T03:53:43.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:53:43.290 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-19T03:53:43.369 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-19T03:53:43.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-19T03:53:43.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:53:43.443 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-19T03:53:43.536 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-19T03:53:43.537 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T03:53:43.547 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T03:53:43.547 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T03:53:43.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T03:53:43.550 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T03:53:43.550 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T03:53:43.552 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T03:53:43.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T03:53:43.580 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T03:53:43.581 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T03:53:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T03:53:43.598 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T03:53:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T03:53:43.619 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T03:53:43.620 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-09-19T03:53:43.625 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-09-19T03:53:43.631 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-09-19T03:53:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-09-19T03:53:43.645 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T03:53:43.651 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-09-19T03:53:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T03:53:43.659 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T03:53:43.662 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-09-19T03:53:43.667 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T03:53:43.677 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T03:53:43.692 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T03:53:43.715 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-19T03:53:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T03:53:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-19T03:53:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.719 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.719 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-09-19T03:53:43.733 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T03:53:43.733 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-09-19T03:53:43.733 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.733 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.733 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.733 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.750 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-19T03:53:43.753 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T03:53:43.753 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-19T03:53:43.754 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.754 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.754 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.754 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T03:53:43.755 INFO:teuthology.run_tasks:Running task install... 2024-09-19T03:53:43.763 DEBUG:teuthology.task.install:project ceph 2024-09-19T03:53:43.763 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T03:53:43.763 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T03:53:43.763 INFO:teuthology.task.install:Using flavor: default 2024-09-19T03:53:43.769 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T03:53:43.769 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T03:53:43.769 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T03:53:43.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:53:43.772 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T03:53:43.772 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:53:43.773 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T03:53:43.773 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:53:43.923 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T03:53:43.923 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T03:53:43.929 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T03:53:43.929 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T03:53:43.932 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T03:53:43.932 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T03:53:44.033 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T03:53:44.033 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:53:44.033 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T03:53:44.039 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T03:53:44.039 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:53:44.039 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T03:53:44.045 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T03:53:44.045 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:53:44.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T03:53:44.060 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T03:53:44.061 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T03:53:44.071 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T03:53:44.072 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T03:53:44.085 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T03:53:44.085 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T03:53:44.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T03:53:44.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T03:53:44.151 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T03:53:44.205 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T03:53:44.213 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T03:53:44.227 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T03:53:44.270 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-09-19T03:53:44.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-09-19T03:53:44.277 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-09-19T03:53:44.278 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-09-19T03:53:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-09-19T03:53:44.294 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-09-19T03:53:44.504 INFO:teuthology.orchestra.run.smithi111.stdout:45 files removed 2024-09-19T03:53:44.515 INFO:teuthology.orchestra.run.smithi019.stdout:45 files removed 2024-09-19T03:53:44.523 INFO:teuthology.orchestra.run.smithi196.stdout:45 files removed 2024-09-19T03:53:44.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T03:53:44.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T03:53:44.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T03:53:45.073 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 319 kB/s | 84 kB 00:00 2024-09-19T03:53:45.113 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-09-19T03:53:45.116 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-19T03:53:45.355 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T03:53:45.397 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T03:53:45.399 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T03:53:45.622 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-19T03:53:45.673 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T03:53:45.675 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T03:53:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-19T03:53:47.450 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-19T03:53:47.597 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.3 MB 00:01 2024-09-19T03:53:49.306 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-19T03:53:51.765 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-19T03:53:53.176 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-19T03:53:54.915 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-19T03:53:56.666 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-19T03:53:57.541 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-19T03:53:58.785 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-19T03:53:59.241 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-19T03:53:59.815 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 23 MB 00:03 2024-09-19T03:54:00.386 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-19T03:54:02.317 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 13 MB/s | 23 MB 00:01 2024-09-19T03:54:02.799 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 23 MB 00:03 2024-09-19T03:54:05.177 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T03:54:07.290 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T03:54:07.291 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T03:54:07.305 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T03:54:07.305 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T03:54:07.306 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T03:54:07.307 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T03:54:07.309 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T03:54:07.312 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T03:54:07.313 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T03:54:07.376 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-19T03:54:07.398 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-19T03:54:07.398 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-19T03:54:07.398 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-19T03:54:07.398 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-19T03:54:07.398 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T03:54:07.399 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T03:54:07.400 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T03:54:07.401 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T03:54:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T03:54:07.403 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T03:54:07.404 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T03:54:07.405 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T03:54:07.406 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T03:54:07.407 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T03:54:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T03:54:07.409 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T03:54:07.410 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T03:54:07.411 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:54:07.412 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-19T03:54:07.412 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-19T03:54:07.412 INFO:teuthology.orchestra.run.smithi111.stdout:Install 148 Packages 2024-09-19T03:54:07.412 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 8 Packages 2024-09-19T03:54:07.412 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:54:07.412 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 226 M 2024-09-19T03:54:07.412 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-19T03:54:07.948 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T03:54:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-19T03:54:08.194 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-19T03:54:08.270 INFO:teuthology.orchestra.run.smithi111.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-19T03:54:08.346 INFO:teuthology.orchestra.run.smithi111.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.9 MB/s | 145 kB 00:00 2024-09-19T03:54:08.421 INFO:teuthology.orchestra.run.smithi111.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T03:54:08.546 INFO:teuthology.orchestra.run.smithi111.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 1.5 MB 00:00 2024-09-19T03:54:08.605 INFO:teuthology.orchestra.run.smithi111.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 9.2 MB/s | 2.4 MB 00:00 2024-09-19T03:54:08.830 INFO:teuthology.orchestra.run.smithi111.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-19T03:54:09.122 INFO:teuthology.orchestra.run.smithi111.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 18 MB/s | 21 MB 00:01 2024-09-19T03:54:09.172 INFO:teuthology.orchestra.run.smithi111.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 506 kB/s | 25 kB 00:00 2024-09-19T03:54:09.440 INFO:teuthology.orchestra.run.smithi111.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 17 MB/s | 11 MB 00:00 2024-09-19T03:54:09.490 INFO:teuthology.orchestra.run.smithi111.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T03:54:09.582 INFO:teuthology.orchestra.run.smithi111.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 295 kB/s | 27 kB 00:00 2024-09-19T03:54:09.749 INFO:teuthology.orchestra.run.smithi111.stdout:(13/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 15 MB/s | 17 MB 00:01 2024-09-19T03:54:09.766 INFO:teuthology.orchestra.run.smithi111.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 173 kB/s | 32 kB 00:00 2024-09-19T03:54:09.799 INFO:teuthology.orchestra.run.smithi111.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 15 MB/s | 751 kB 00:00 2024-09-19T03:54:09.825 INFO:teuthology.orchestra.run.smithi111.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T03:54:09.850 INFO:teuthology.orchestra.run.smithi111.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T03:54:09.875 INFO:teuthology.orchestra.run.smithi111.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 9.9 MB/s | 506 kB 00:00 2024-09-19T03:54:09.925 INFO:teuthology.orchestra.run.smithi111.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 906 kB/s | 45 kB 00:00 2024-09-19T03:54:10.017 INFO:teuthology.orchestra.run.smithi111.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T03:54:10.126 INFO:teuthology.orchestra.run.smithi111.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-19T03:54:10.160 INFO:teuthology.orchestra.run.smithi111.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.1 MB/s | 161 kB 00:00 2024-09-19T03:54:10.202 INFO:teuthology.orchestra.run.smithi111.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T03:54:10.234 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T03:54:10.235 INFO:teuthology.orchestra.run.smithi111.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.9 MB/s | 300 kB 00:00 2024-09-19T03:54:10.235 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T03:54:10.250 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T03:54:10.250 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T03:54:10.251 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T03:54:10.253 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T03:54:10.255 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T03:54:10.257 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T03:54:10.259 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T03:54:10.260 INFO:teuthology.orchestra.run.smithi111.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T03:54:10.286 INFO:teuthology.orchestra.run.smithi111.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T03:54:10.322 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T03:54:10.322 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T03:54:10.326 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-19T03:54:10.334 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T03:54:10.335 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T03:54:10.335 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T03:54:10.336 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T03:54:10.338 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T03:54:10.340 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T03:54:10.341 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T03:54:10.344 INFO:teuthology.orchestra.run.smithi111.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T03:54:10.349 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T03:54:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T03:54:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T03:54:10.352 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T03:54:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T03:54:10.354 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T03:54:10.355 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T03:54:10.355 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T03:54:10.355 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T03:54:10.355 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T03:54:10.355 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T03:54:10.355 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T03:54:10.355 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T03:54:10.355 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T03:54:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T03:54:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T03:54:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T03:54:10.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T03:54:10.360 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T03:54:10.361 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T03:54:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 8 Packages 2024-09-19T03:54:10.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:54:10.364 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 226 M 2024-09-19T03:54:10.364 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-19T03:54:10.404 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-19T03:54:10.411 INFO:teuthology.orchestra.run.smithi111.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T03:54:10.419 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T03:54:10.420 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T03:54:10.421 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T03:54:10.422 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T03:54:10.423 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T03:54:10.424 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T03:54:10.425 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T03:54:10.426 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T03:54:10.427 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T03:54:10.428 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T03:54:10.429 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T03:54:10.430 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T03:54:10.431 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-09-19T03:54:10.432 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T03:54:10.432 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:54:10.432 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-19T03:54:10.432 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-19T03:54:10.432 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-09-19T03:54:10.432 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 8 Packages 2024-09-19T03:54:10.432 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:54:10.433 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 226 M 2024-09-19T03:54:10.433 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-19T03:54:10.437 INFO:teuthology.orchestra.run.smithi111.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 346 kB/s | 32 kB 00:00 2024-09-19T03:54:10.462 INFO:teuthology.orchestra.run.smithi111.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T03:54:10.737 INFO:teuthology.orchestra.run.smithi111.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 12 MB/s | 3.6 MB 00:00 2024-09-19T03:54:10.829 INFO:teuthology.orchestra.run.smithi111.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-19T03:54:10.854 INFO:teuthology.orchestra.run.smithi111.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.1 MB/s | 249 kB 00:00 2024-09-19T03:54:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 833 kB/s | 49 kB 00:00 2024-09-19T03:54:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 277 kB/s | 16 kB 00:00 2024-09-19T03:54:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T03:54:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 437 kB/s | 18 kB 00:00 2024-09-19T03:54:11.079 INFO:teuthology.orchestra.run.smithi196.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-19T03:54:11.081 INFO:teuthology.orchestra.run.smithi111.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 4.5 MB/s | 765 kB 00:00 2024-09-19T03:54:11.123 INFO:teuthology.orchestra.run.smithi019.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-19T03:54:11.313 INFO:teuthology.orchestra.run.smithi196.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.8 MB/s | 896 kB 00:00 2024-09-19T03:54:11.388 INFO:teuthology.orchestra.run.smithi196.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.9 MB/s | 145 kB 00:00 2024-09-19T03:54:11.490 INFO:teuthology.orchestra.run.smithi019.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.4 MB/s | 896 kB 00:00 2024-09-19T03:54:11.496 INFO:teuthology.orchestra.run.smithi196.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 9.7 MB/s | 5.4 MB 00:00 2024-09-19T03:54:11.540 INFO:teuthology.orchestra.run.smithi019.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.8 MB/s | 145 kB 00:00 2024-09-19T03:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 20 MB/s | 49 MB 00:02 2024-09-19T03:54:11.647 INFO:teuthology.orchestra.run.smithi196.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 10 MB/s | 1.5 MB 00:00 2024-09-19T03:54:11.739 INFO:teuthology.orchestra.run.smithi196.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-09-19T03:54:12.066 INFO:teuthology.orchestra.run.smithi111.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 74 kB/s | 80 kB 00:01 2024-09-19T03:54:12.090 INFO:teuthology.orchestra.run.smithi019.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 4.9 MB/s | 5.4 MB 00:01 2024-09-19T03:54:12.182 INFO:teuthology.orchestra.run.smithi019.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.7 MB/s | 2.4 MB 00:00 2024-09-19T03:54:12.198 INFO:teuthology.orchestra.run.smithi196.stdout:(7/156): ceph-common-19.1.1-361.g2f599a46.el9.x 17 MB/s | 21 MB 00:01 2024-09-19T03:54:12.264 INFO:teuthology.orchestra.run.smithi196.stdout:(8/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 7.6 MB/s | 4.7 MB 00:00 2024-09-19T03:54:12.283 INFO:teuthology.orchestra.run.smithi019.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 7.9 MB/s | 1.5 MB 00:00 2024-09-19T03:54:12.315 INFO:teuthology.orchestra.run.smithi196.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 505 kB/s | 25 kB 00:00 2024-09-19T03:54:12.341 INFO:teuthology.orchestra.run.smithi111.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 244 kB/s | 191 kB 00:00 2024-09-19T03:54:12.375 INFO:teuthology.orchestra.run.smithi111.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 620 kB/s | 801 kB 00:01 2024-09-19T03:54:12.815 INFO:teuthology.orchestra.run.smithi196.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 17 MB/s | 11 MB 00:00 2024-09-19T03:54:12.850 INFO:teuthology.orchestra.run.smithi111.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 323 kB/s | 253 kB 00:00 2024-09-19T03:54:12.899 INFO:teuthology.orchestra.run.smithi196.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 2.7 MB/s | 228 kB 00:00 2024-09-19T03:54:13.042 INFO:teuthology.orchestra.run.smithi111.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-19T03:54:13.116 INFO:teuthology.orchestra.run.smithi196.stdout:(12/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 17 MB 00:01 2024-09-19T03:54:13.141 INFO:teuthology.orchestra.run.smithi196.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 112 kB/s | 27 kB 00:00 2024-09-19T03:54:13.168 INFO:teuthology.orchestra.run.smithi196.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 611 kB/s | 32 kB 00:00 2024-09-19T03:54:13.201 INFO:teuthology.orchestra.run.smithi196.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 751 kB 00:00 2024-09-19T03:54:13.227 INFO:teuthology.orchestra.run.smithi196.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T03:54:13.241 INFO:teuthology.orchestra.run.smithi019.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 4.4 MB/s | 4.7 MB 00:01 2024-09-19T03:54:13.252 INFO:teuthology.orchestra.run.smithi196.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T03:54:13.277 INFO:teuthology.orchestra.run.smithi196.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 9.8 MB/s | 506 kB 00:00 2024-09-19T03:54:13.344 INFO:teuthology.orchestra.run.smithi196.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 678 kB/s | 45 kB 00:00 2024-09-19T03:54:13.358 INFO:teuthology.orchestra.run.smithi111.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-09-19T03:54:13.437 INFO:teuthology.orchestra.run.smithi196.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T03:54:13.542 INFO:teuthology.orchestra.run.smithi111.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 195 kB/s | 135 kB 00:00 2024-09-19T03:54:13.562 INFO:teuthology.orchestra.run.smithi196.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-19T03:54:13.587 INFO:teuthology.orchestra.run.smithi196.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.0 MB/s | 161 kB 00:00 2024-09-19T03:54:13.637 INFO:teuthology.orchestra.run.smithi196.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T03:54:13.662 INFO:teuthology.orchestra.run.smithi196.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.9 MB/s | 300 kB 00:00 2024-09-19T03:54:13.696 INFO:teuthology.orchestra.run.smithi196.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T03:54:13.730 INFO:teuthology.orchestra.run.smithi196.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-19T03:54:13.751 INFO:teuthology.orchestra.run.smithi111.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-09-19T03:54:13.839 INFO:teuthology.orchestra.run.smithi196.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.5 MB/s | 171 kB 00:00 2024-09-19T03:54:13.867 INFO:teuthology.orchestra.run.smithi111.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-19T03:54:13.926 INFO:teuthology.orchestra.run.smithi111.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-19T03:54:13.930 INFO:teuthology.orchestra.run.smithi196.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-19T03:54:13.956 INFO:teuthology.orchestra.run.smithi196.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 272 kB/s | 32 kB 00:00 2024-09-19T03:54:13.985 INFO:teuthology.orchestra.run.smithi111.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-19T03:54:13.989 INFO:teuthology.orchestra.run.smithi196.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.4 MB/s | 146 kB 00:00 2024-09-19T03:54:14.052 INFO:teuthology.orchestra.run.smithi111.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-19T03:54:14.077 INFO:teuthology.orchestra.run.smithi111.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-19T03:54:14.102 INFO:teuthology.orchestra.run.smithi111.stdout:(53/156): python3-urllib3-1.26.5-6.el9.noarch.r 289 kB/s | 215 kB 00:00 2024-09-19T03:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-09-19T03:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 192 kB/s | 11 kB 00:00 2024-09-19T03:54:14.231 INFO:teuthology.orchestra.run.smithi196.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 13 MB/s | 3.6 MB 00:00 2024-09-19T03:54:14.236 INFO:teuthology.orchestra.run.smithi111.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 18 kB 00:00 2024-09-19T03:54:14.278 INFO:teuthology.orchestra.run.smithi111.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-19T03:54:14.307 INFO:teuthology.orchestra.run.smithi196.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.3 MB/s | 249 kB 00:00 2024-09-19T03:54:14.320 INFO:teuthology.orchestra.run.smithi111.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-19T03:54:14.357 INFO:teuthology.orchestra.run.smithi196.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 990 kB/s | 49 kB 00:00 2024-09-19T03:54:14.379 INFO:teuthology.orchestra.run.smithi111.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-09-19T03:54:14.446 INFO:teuthology.orchestra.run.smithi111.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-19T03:54:14.474 INFO:teuthology.orchestra.run.smithi196.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-19T03:54:14.491 INFO:teuthology.orchestra.run.smithi196.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 123 kB/s | 16 kB 00:00 2024-09-19T03:54:14.491 INFO:teuthology.orchestra.run.smithi019.stdout:(8/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86_ 7.6 MB/s | 17 MB 00:02 2024-09-19T03:54:14.504 INFO:teuthology.orchestra.run.smithi111.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-19T03:54:14.530 INFO:teuthology.orchestra.run.smithi111.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-09-19T03:54:14.533 INFO:teuthology.orchestra.run.smithi196.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 285 kB 00:00 2024-09-19T03:54:14.541 INFO:teuthology.orchestra.run.smithi019.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 506 kB/s | 25 kB 00:00 2024-09-19T03:54:14.575 INFO:teuthology.orchestra.run.smithi196.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 8.9 MB/s | 765 kB 00:00 2024-09-19T03:54:14.588 INFO:teuthology.orchestra.run.smithi111.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-19T03:54:14.600 INFO:teuthology.orchestra.run.smithi196.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 272 kB/s | 18 kB 00:00 2024-09-19T03:54:14.647 INFO:teuthology.orchestra.run.smithi111.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 661 kB/s | 38 kB 00:00 2024-09-19T03:54:14.680 INFO:teuthology.orchestra.run.smithi111.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-19T03:54:15.034 INFO:teuthology.orchestra.run.smithi019.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 5.9 MB/s | 11 MB 00:01 2024-09-19T03:54:15.084 INFO:teuthology.orchestra.run.smithi019.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T03:54:15.123 INFO:teuthology.orchestra.run.smithi111.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-19T03:54:15.134 INFO:teuthology.orchestra.run.smithi019.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 540 kB/s | 27 kB 00:00 2024-09-19T03:54:15.185 INFO:teuthology.orchestra.run.smithi019.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 637 kB/s | 32 kB 00:00 2024-09-19T03:54:15.192 INFO:teuthology.orchestra.run.smithi196.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 17 MB/s | 49 MB 00:02 2024-09-19T03:54:15.218 INFO:teuthology.orchestra.run.smithi196.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 124 kB/s | 80 kB 00:00 2024-09-19T03:54:15.260 INFO:teuthology.orchestra.run.smithi196.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-19T03:54:15.293 INFO:teuthology.orchestra.run.smithi196.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-19T03:54:15.336 INFO:teuthology.orchestra.run.smithi196.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-19T03:54:15.360 INFO:teuthology.orchestra.run.smithi196.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-19T03:54:15.386 INFO:teuthology.orchestra.run.smithi196.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T03:54:15.410 INFO:teuthology.orchestra.run.smithi019.stdout:(14/156): ceph-common-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 21 MB 00:04 2024-09-19T03:54:15.411 INFO:teuthology.orchestra.run.smithi196.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T03:54:15.436 INFO:teuthology.orchestra.run.smithi196.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-19T03:54:15.443 INFO:teuthology.orchestra.run.smithi019.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 2.8 MB/s | 751 kB 00:00 2024-09-19T03:54:15.469 INFO:teuthology.orchestra.run.smithi019.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T03:54:15.470 INFO:teuthology.orchestra.run.smithi196.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-09-19T03:54:15.494 INFO:teuthology.orchestra.run.smithi019.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T03:54:15.548 INFO:teuthology.orchestra.run.smithi111.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-19T03:54:15.569 INFO:teuthology.orchestra.run.smithi019.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 4.9 MB/s | 506 kB 00:00 2024-09-19T03:54:15.619 INFO:teuthology.orchestra.run.smithi019.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 905 kB/s | 45 kB 00:00 2024-09-19T03:54:15.645 INFO:teuthology.orchestra.run.smithi196.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-09-19T03:54:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.7 MB/s | 136 kB 00:00 2024-09-19T03:54:15.704 INFO:teuthology.orchestra.run.smithi196.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-19T03:54:15.720 INFO:teuthology.orchestra.run.smithi019.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T03:54:15.729 INFO:teuthology.orchestra.run.smithi196.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-19T03:54:15.763 INFO:teuthology.orchestra.run.smithi196.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-19T03:54:15.787 INFO:teuthology.orchestra.run.smithi019.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.7 MB/s | 320 kB 00:00 2024-09-19T03:54:15.788 INFO:teuthology.orchestra.run.smithi196.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-09-19T03:54:15.798 INFO:teuthology.orchestra.run.smithi111.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.3 MB/s | 4.9 MB 00:01 2024-09-19T03:54:15.813 INFO:teuthology.orchestra.run.smithi196.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-19T03:54:15.854 INFO:teuthology.orchestra.run.smithi019.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 4.4 MB/s | 300 kB 00:00 2024-09-19T03:54:15.856 INFO:teuthology.orchestra.run.smithi111.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 2.9 MB/s | 910 kB 00:00 2024-09-19T03:54:15.872 INFO:teuthology.orchestra.run.smithi196.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-09-19T03:54:15.897 INFO:teuthology.orchestra.run.smithi196.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-19T03:54:15.899 INFO:teuthology.orchestra.run.smithi111.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-19T03:54:15.904 INFO:teuthology.orchestra.run.smithi019.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T03:54:15.931 INFO:teuthology.orchestra.run.smithi196.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T03:54:15.956 INFO:teuthology.orchestra.run.smithi019.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T03:54:15.956 INFO:teuthology.orchestra.run.smithi196.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-19T03:54:16.048 INFO:teuthology.orchestra.run.smithi196.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-19T03:54:16.073 INFO:teuthology.orchestra.run.smithi196.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-19T03:54:16.082 INFO:teuthology.orchestra.run.smithi111.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-09-19T03:54:16.099 INFO:teuthology.orchestra.run.smithi196.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-19T03:54:16.113 INFO:teuthology.orchestra.run.smithi019.stdout:(26/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 8.6 MB/s | 5.3 MB 00:00 2024-09-19T03:54:16.124 INFO:teuthology.orchestra.run.smithi196.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-19T03:54:16.141 INFO:teuthology.orchestra.run.smithi111.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T03:54:16.158 INFO:teuthology.orchestra.run.smithi196.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-19T03:54:16.163 INFO:teuthology.orchestra.run.smithi019.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-19T03:54:16.183 INFO:teuthology.orchestra.run.smithi196.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-19T03:54:16.205 INFO:teuthology.orchestra.run.smithi019.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 764 kB/s | 32 kB 00:00 2024-09-19T03:54:16.234 INFO:teuthology.orchestra.run.smithi196.stdout:(65/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.4 MB/s | 3.0 MB 00:00 2024-09-19T03:54:16.256 INFO:teuthology.orchestra.run.smithi019.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T03:54:16.259 INFO:teuthology.orchestra.run.smithi196.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 382 kB/s | 38 kB 00:00 2024-09-19T03:54:16.275 INFO:teuthology.orchestra.run.smithi111.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-09-19T03:54:16.308 INFO:teuthology.orchestra.run.smithi111.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-19T03:54:16.392 INFO:teuthology.orchestra.run.smithi111.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-19T03:54:16.414 INFO:teuthology.orchestra.run.smithi019.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.8 MB/s | 3.1 MB 00:00 2024-09-19T03:54:16.442 INFO:teuthology.orchestra.run.smithi196.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-09-19T03:54:16.475 INFO:teuthology.orchestra.run.smithi111.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-19T03:54:16.509 INFO:teuthology.orchestra.run.smithi111.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 775 kB/s | 26 kB 00:00 2024-09-19T03:54:16.543 INFO:teuthology.orchestra.run.smithi111.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 577 kB/s | 19 kB 00:00 2024-09-19T03:54:16.568 INFO:teuthology.orchestra.run.smithi196.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-19T03:54:16.618 INFO:teuthology.orchestra.run.smithi111.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-09-19T03:54:16.656 INFO:teuthology.orchestra.run.smithi019.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 8.9 MB/s | 3.6 MB 00:00 2024-09-19T03:54:16.660 INFO:teuthology.orchestra.run.smithi111.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 837 kB/s | 35 kB 00:00 2024-09-19T03:54:16.693 INFO:teuthology.orchestra.run.smithi196.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-09-19T03:54:16.715 INFO:teuthology.orchestra.run.smithi019.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.2 MB/s | 249 kB 00:00 2024-09-19T03:54:16.768 INFO:teuthology.orchestra.run.smithi019.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 987 kB/s | 49 kB 00:00 2024-09-19T03:54:16.815 INFO:teuthology.orchestra.run.smithi019.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 330 kB/s | 16 kB 00:00 2024-09-19T03:54:16.826 INFO:teuthology.orchestra.run.smithi196.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 531 kB/s | 137 kB 00:00 2024-09-19T03:54:16.882 INFO:teuthology.orchestra.run.smithi019.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.2 MB/s | 285 kB 00:00 2024-09-19T03:54:16.893 INFO:teuthology.orchestra.run.smithi196.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 7.0 MB/s | 4.9 MB 00:00 2024-09-19T03:54:16.952 INFO:teuthology.orchestra.run.smithi196.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T03:54:16.983 INFO:teuthology.orchestra.run.smithi019.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 7.5 MB/s | 765 kB 00:00 2024-09-19T03:54:17.019 INFO:teuthology.orchestra.run.smithi196.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-09-19T03:54:17.033 INFO:teuthology.orchestra.run.smithi019.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 365 kB/s | 18 kB 00:00 2024-09-19T03:54:17.078 INFO:teuthology.orchestra.run.smithi196.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-19T03:54:17.136 INFO:teuthology.orchestra.run.smithi196.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-19T03:54:17.212 INFO:teuthology.orchestra.run.smithi196.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-09-19T03:54:17.245 INFO:teuthology.orchestra.run.smithi196.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-19T03:54:17.271 INFO:teuthology.orchestra.run.smithi196.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-09-19T03:54:17.292 INFO:teuthology.orchestra.run.smithi019.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 8.4 MB/s | 7.4 MB 00:00 2024-09-19T03:54:17.304 INFO:teuthology.orchestra.run.smithi196.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-09-19T03:54:17.329 INFO:teuthology.orchestra.run.smithi196.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-19T03:54:17.355 INFO:teuthology.orchestra.run.smithi196.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-19T03:54:17.413 INFO:teuthology.orchestra.run.smithi196.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-19T03:54:17.492 INFO:teuthology.orchestra.run.smithi019.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-19T03:54:17.522 INFO:teuthology.orchestra.run.smithi196.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-19T03:54:17.547 INFO:teuthology.orchestra.run.smithi196.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-09-19T03:54:17.572 INFO:teuthology.orchestra.run.smithi196.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T03:54:17.598 INFO:teuthology.orchestra.run.smithi196.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-19T03:54:17.618 INFO:teuthology.orchestra.run.smithi111.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-09-19T03:54:17.665 INFO:teuthology.orchestra.run.smithi196.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-19T03:54:17.690 INFO:teuthology.orchestra.run.smithi196.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-19T03:54:17.702 INFO:teuthology.orchestra.run.smithi111.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-19T03:54:17.723 INFO:teuthology.orchestra.run.smithi196.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-09-19T03:54:17.749 INFO:teuthology.orchestra.run.smithi196.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-09-19T03:54:17.774 INFO:teuthology.orchestra.run.smithi196.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-19T03:54:17.784 INFO:teuthology.orchestra.run.smithi019.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 655 kB/s | 191 kB 00:00 2024-09-19T03:54:17.785 INFO:teuthology.orchestra.run.smithi111.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-19T03:54:17.808 INFO:teuthology.orchestra.run.smithi196.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 638 kB/s | 54 kB 00:00 2024-09-19T03:54:17.833 INFO:teuthology.orchestra.run.smithi196.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 707 kB/s | 42 kB 00:00 2024-09-19T03:54:17.844 INFO:teuthology.orchestra.run.smithi111.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T03:54:17.867 INFO:teuthology.orchestra.run.smithi196.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-09-19T03:54:17.892 INFO:teuthology.orchestra.run.smithi196.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T03:54:17.918 INFO:teuthology.orchestra.run.smithi196.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-19T03:54:17.919 INFO:teuthology.orchestra.run.smithi111.stdout:(85/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-09-19T03:54:17.943 INFO:teuthology.orchestra.run.smithi196.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-19T03:54:17.944 INFO:teuthology.orchestra.run.smithi111.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-09-19T03:54:17.985 INFO:teuthology.orchestra.run.smithi019.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-19T03:54:17.985 INFO:teuthology.orchestra.run.smithi196.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-19T03:54:18.010 INFO:teuthology.orchestra.run.smithi196.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 864 kB/s | 58 kB 00:00 2024-09-19T03:54:18.012 INFO:teuthology.orchestra.run.smithi111.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-19T03:54:18.044 INFO:teuthology.orchestra.run.smithi196.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T03:54:18.054 INFO:teuthology.orchestra.run.smithi111.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-09-19T03:54:18.069 INFO:teuthology.orchestra.run.smithi196.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T03:54:18.095 INFO:teuthology.orchestra.run.smithi111.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-19T03:54:18.103 INFO:teuthology.orchestra.run.smithi196.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-09-19T03:54:18.137 INFO:teuthology.orchestra.run.smithi111.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-09-19T03:54:18.204 INFO:teuthology.orchestra.run.smithi111.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.3 MB/s | 5.1 MB 00:01 2024-09-19T03:54:18.229 INFO:teuthology.orchestra.run.smithi111.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 401 kB/s | 54 kB 00:00 2024-09-19T03:54:18.243 INFO:teuthology.orchestra.run.smithi019.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 4.8 MB/s | 1.2 MB 00:00 2024-09-19T03:54:18.271 INFO:teuthology.orchestra.run.smithi111.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-19T03:54:18.285 INFO:teuthology.orchestra.run.smithi019.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 807 kB/s | 801 kB 00:00 2024-09-19T03:54:18.297 INFO:teuthology.orchestra.run.smithi111.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-09-19T03:54:18.339 INFO:teuthology.orchestra.run.smithi111.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-09-19T03:54:18.353 INFO:teuthology.orchestra.run.smithi196.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-09-19T03:54:18.381 INFO:teuthology.orchestra.run.smithi111.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-19T03:54:18.403 INFO:teuthology.orchestra.run.smithi196.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T03:54:18.406 INFO:teuthology.orchestra.run.smithi111.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-19T03:54:18.437 INFO:teuthology.orchestra.run.smithi196.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T03:54:18.456 INFO:teuthology.orchestra.run.smithi111.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T03:54:18.462 INFO:teuthology.orchestra.run.smithi196.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-09-19T03:54:18.515 INFO:teuthology.orchestra.run.smithi111.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-19T03:54:18.540 INFO:teuthology.orchestra.run.smithi111.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-19T03:54:18.579 INFO:teuthology.orchestra.run.smithi196.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-19T03:54:18.582 INFO:teuthology.orchestra.run.smithi111.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 949 kB/s | 64 kB 00:00 2024-09-19T03:54:18.604 INFO:teuthology.orchestra.run.smithi196.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T03:54:18.607 INFO:teuthology.orchestra.run.smithi111.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-09-19T03:54:18.630 INFO:teuthology.orchestra.run.smithi196.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T03:54:18.663 INFO:teuthology.orchestra.run.smithi196.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-19T03:54:18.716 INFO:teuthology.orchestra.run.smithi111.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-09-19T03:54:18.752 INFO:teuthology.orchestra.run.smithi019.stdout:(44/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 12 MB/s | 49 MB 00:04 2024-09-19T03:54:18.777 INFO:teuthology.orchestra.run.smithi019.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 199 kB/s | 106 kB 00:00 2024-09-19T03:54:18.791 INFO:teuthology.orchestra.run.smithi111.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-19T03:54:18.802 INFO:teuthology.orchestra.run.smithi019.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 261 kB/s | 135 kB 00:00 2024-09-19T03:54:18.817 INFO:teuthology.orchestra.run.smithi111.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-19T03:54:18.822 INFO:teuthology.orchestra.run.smithi196.stdout:(111/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-19T03:54:18.842 INFO:teuthology.orchestra.run.smithi111.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-19T03:54:18.856 INFO:teuthology.orchestra.run.smithi196.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 788 kB/s | 151 kB 00:00 2024-09-19T03:54:18.886 INFO:teuthology.orchestra.run.smithi019.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-19T03:54:18.898 INFO:teuthology.orchestra.run.smithi196.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 14 MB/s | 548 kB 00:00 2024-09-19T03:54:18.901 INFO:teuthology.orchestra.run.smithi111.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-09-19T03:54:18.923 INFO:teuthology.orchestra.run.smithi196.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T03:54:18.948 INFO:teuthology.orchestra.run.smithi196.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T03:54:18.969 INFO:teuthology.orchestra.run.smithi019.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-19T03:54:18.973 INFO:teuthology.orchestra.run.smithi196.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-19T03:54:18.984 INFO:teuthology.orchestra.run.smithi111.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-09-19T03:54:18.999 INFO:teuthology.orchestra.run.smithi196.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-19T03:54:19.009 INFO:teuthology.orchestra.run.smithi111.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-09-19T03:54:19.012 INFO:teuthology.orchestra.run.smithi019.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-19T03:54:19.035 INFO:teuthology.orchestra.run.smithi111.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-19T03:54:19.041 INFO:teuthology.orchestra.run.smithi196.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-09-19T03:54:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-19T03:54:19.060 INFO:teuthology.orchestra.run.smithi111.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T03:54:19.066 INFO:teuthology.orchestra.run.smithi196.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T03:54:19.085 INFO:teuthology.orchestra.run.smithi111.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-19T03:54:19.092 INFO:teuthology.orchestra.run.smithi196.stdout:(120/156): protobuf-compiler-3.14.0-13.el9.x86_ 873 kB/s | 863 kB 00:00 2024-09-19T03:54:19.119 INFO:teuthology.orchestra.run.smithi111.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-19T03:54:19.125 INFO:teuthology.orchestra.run.smithi196.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-19T03:54:19.144 INFO:teuthology.orchestra.run.smithi111.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T03:54:19.151 INFO:teuthology.orchestra.run.smithi196.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-19T03:54:19.154 INFO:teuthology.orchestra.run.smithi019.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-19T03:54:19.170 INFO:teuthology.orchestra.run.smithi111.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T03:54:19.179 INFO:teuthology.orchestra.run.smithi019.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-19T03:54:19.184 INFO:teuthology.orchestra.run.smithi196.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 324 kB/s | 11 kB 00:00 2024-09-19T03:54:19.195 INFO:teuthology.orchestra.run.smithi111.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-19T03:54:19.204 INFO:teuthology.orchestra.run.smithi019.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-09-19T03:54:19.218 INFO:teuthology.orchestra.run.smithi196.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-09-19T03:54:19.229 INFO:teuthology.orchestra.run.smithi111.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-19T03:54:19.243 INFO:teuthology.orchestra.run.smithi196.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-09-19T03:54:19.246 INFO:teuthology.orchestra.run.smithi019.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-19T03:54:19.268 INFO:teuthology.orchestra.run.smithi196.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-19T03:54:19.270 INFO:teuthology.orchestra.run.smithi111.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-09-19T03:54:19.272 INFO:teuthology.orchestra.run.smithi019.stdout:(55/156): python3-requests-2.25.1-8.el9.noarch. 240 kB/s | 125 kB 00:00 2024-09-19T03:54:19.293 INFO:teuthology.orchestra.run.smithi196.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-19T03:54:19.296 INFO:teuthology.orchestra.run.smithi111.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T03:54:19.297 INFO:teuthology.orchestra.run.smithi019.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-19T03:54:19.319 INFO:teuthology.orchestra.run.smithi196.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T03:54:19.329 INFO:teuthology.orchestra.run.smithi111.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-19T03:54:19.339 INFO:teuthology.orchestra.run.smithi019.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-19T03:54:19.353 INFO:teuthology.orchestra.run.smithi196.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T03:54:19.355 INFO:teuthology.orchestra.run.smithi111.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T03:54:19.378 INFO:teuthology.orchestra.run.smithi196.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T03:54:19.380 INFO:teuthology.orchestra.run.smithi111.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-19T03:54:19.381 INFO:teuthology.orchestra.run.smithi019.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-19T03:54:19.403 INFO:teuthology.orchestra.run.smithi196.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T03:54:19.405 INFO:teuthology.orchestra.run.smithi111.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-19T03:54:19.437 INFO:teuthology.orchestra.run.smithi196.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-09-19T03:54:19.439 INFO:teuthology.orchestra.run.smithi111.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 532 kB/s | 18 kB 00:00 2024-09-19T03:54:19.462 INFO:teuthology.orchestra.run.smithi196.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-09-19T03:54:19.464 INFO:teuthology.orchestra.run.smithi111.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-09-19T03:54:19.489 INFO:teuthology.orchestra.run.smithi111.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-19T03:54:19.498 INFO:teuthology.orchestra.run.smithi019.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-09-19T03:54:19.515 INFO:teuthology.orchestra.run.smithi111.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-19T03:54:19.516 INFO:teuthology.orchestra.run.smithi196.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-19T03:54:19.540 INFO:teuthology.orchestra.run.smithi019.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-19T03:54:19.546 INFO:teuthology.orchestra.run.smithi196.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-09-19T03:54:19.565 INFO:teuthology.orchestra.run.smithi019.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-19T03:54:19.565 INFO:teuthology.orchestra.run.smithi111.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-19T03:54:19.572 INFO:teuthology.orchestra.run.smithi196.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 516 kB/s | 28 kB 00:00 2024-09-19T03:54:19.590 INFO:teuthology.orchestra.run.smithi019.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-19T03:54:19.591 INFO:teuthology.orchestra.run.smithi111.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T03:54:19.597 INFO:teuthology.orchestra.run.smithi196.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-19T03:54:19.615 INFO:teuthology.orchestra.run.smithi019.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-19T03:54:19.616 INFO:teuthology.orchestra.run.smithi111.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T03:54:19.622 INFO:teuthology.orchestra.run.smithi196.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T03:54:19.641 INFO:teuthology.orchestra.run.smithi111.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T03:54:19.647 INFO:teuthology.orchestra.run.smithi196.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T03:54:19.657 INFO:teuthology.orchestra.run.smithi019.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 928 kB/s | 38 kB 00:00 2024-09-19T03:54:19.666 INFO:teuthology.orchestra.run.smithi111.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-19T03:54:19.673 INFO:teuthology.orchestra.run.smithi196.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T03:54:19.691 INFO:teuthology.orchestra.run.smithi019.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-19T03:54:19.698 INFO:teuthology.orchestra.run.smithi196.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-19T03:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-19T03:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-19T03:54:19.732 INFO:teuthology.orchestra.run.smithi196.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-19T03:54:19.751 INFO:teuthology.orchestra.run.smithi111.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T03:54:19.773 INFO:teuthology.orchestra.run.smithi196.stdout:(143/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-09-19T03:54:19.776 INFO:teuthology.orchestra.run.smithi111.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-19T03:54:19.799 INFO:teuthology.orchestra.run.smithi196.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-19T03:54:19.801 INFO:teuthology.orchestra.run.smithi111.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T03:54:19.824 INFO:teuthology.orchestra.run.smithi196.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 807 kB/s | 20 kB 00:00 2024-09-19T03:54:19.827 INFO:teuthology.orchestra.run.smithi111.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-19T03:54:19.849 INFO:teuthology.orchestra.run.smithi196.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T03:54:19.852 INFO:teuthology.orchestra.run.smithi111.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-19T03:54:19.875 INFO:teuthology.orchestra.run.smithi196.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-19T03:54:19.877 INFO:teuthology.orchestra.run.smithi111.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-19T03:54:19.902 INFO:teuthology.orchestra.run.smithi111.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-19T03:54:19.925 INFO:teuthology.orchestra.run.smithi196.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T03:54:19.928 INFO:teuthology.orchestra.run.smithi111.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T03:54:19.953 INFO:teuthology.orchestra.run.smithi111.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T03:54:19.978 INFO:teuthology.orchestra.run.smithi111.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-19T03:54:20.004 INFO:teuthology.orchestra.run.smithi111.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-19T03:54:20.008 INFO:teuthology.orchestra.run.smithi019.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-09-19T03:54:20.029 INFO:teuthology.orchestra.run.smithi111.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T03:54:20.079 INFO:teuthology.orchestra.run.smithi111.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-19T03:54:20.109 INFO:teuthology.orchestra.run.smithi196.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-19T03:54:20.151 INFO:teuthology.orchestra.run.smithi196.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-09-19T03:54:20.236 INFO:teuthology.orchestra.run.smithi196.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 486 kB/s | 41 kB 00:00 2024-09-19T03:54:20.293 INFO:teuthology.orchestra.run.smithi196.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 9.8 MB/s | 1.8 MB 00:00 2024-09-19T03:54:20.305 INFO:teuthology.orchestra.run.smithi111.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T03:54:20.318 INFO:teuthology.orchestra.run.smithi196.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 3.0 MB/s | 249 kB 00:00 2024-09-19T03:54:20.360 INFO:teuthology.orchestra.run.smithi196.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-19T03:54:20.411 INFO:teuthology.orchestra.run.smithi196.stdout:(155/156): librbd1-19.1.1-361.g2f599a46.el9.x86 5.9 MB/s | 3.2 MB 00:00 2024-09-19T03:54:20.452 INFO:teuthology.orchestra.run.smithi196.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-19T03:54:20.459 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:54:20.459 INFO:teuthology.orchestra.run.smithi196.stdout:Total 23 MB/s | 226 MB 00:10 2024-09-19T03:54:20.467 INFO:teuthology.orchestra.run.smithi019.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-09-19T03:54:20.530 INFO:teuthology.orchestra.run.smithi111.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.0 MB/s | 3.2 MB 00:00 2024-09-19T03:54:20.726 INFO:teuthology.orchestra.run.smithi019.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.4 MB/s | 910 kB 00:00 2024-09-19T03:54:20.818 INFO:teuthology.orchestra.run.smithi019.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-19T03:54:20.843 INFO:teuthology.orchestra.run.smithi019.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-19T03:54:21.018 INFO:teuthology.orchestra.run.smithi019.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-19T03:54:21.077 INFO:teuthology.orchestra.run.smithi019.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T03:54:21.219 INFO:teuthology.orchestra.run.smithi019.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-09-19T03:54:21.252 INFO:teuthology.orchestra.run.smithi019.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-19T03:54:21.330 INFO:teuthology.orchestra.run.smithi111.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-09-19T03:54:21.336 INFO:teuthology.orchestra.run.smithi019.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-19T03:54:21.419 INFO:teuthology.orchestra.run.smithi019.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-19T03:54:21.461 INFO:teuthology.orchestra.run.smithi019.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-09-19T03:54:21.495 INFO:teuthology.orchestra.run.smithi019.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 575 kB/s | 19 kB 00:00 2024-09-19T03:54:21.570 INFO:teuthology.orchestra.run.smithi019.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-19T03:54:21.589 INFO:teuthology.orchestra.run.smithi111.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-09-19T03:54:21.612 INFO:teuthology.orchestra.run.smithi019.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 836 kB/s | 35 kB 00:00 2024-09-19T03:54:21.695 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-19T03:54:21.847 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-19T03:54:21.847 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-19T03:54:21.898 INFO:teuthology.orchestra.run.smithi111.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 72 kB/s | 41 kB 00:00 2024-09-19T03:54:22.364 INFO:teuthology.orchestra.run.smithi111.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 321 kB/s | 249 kB 00:00 2024-09-19T03:54:22.487 INFO:teuthology.orchestra.run.smithi019.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.5 MB/s | 5.8 MB 00:01 2024-09-19T03:54:22.579 INFO:teuthology.orchestra.run.smithi019.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-19T03:54:22.671 INFO:teuthology.orchestra.run.smithi019.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-19T03:54:22.730 INFO:teuthology.orchestra.run.smithi019.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T03:54:22.822 INFO:teuthology.orchestra.run.smithi019.stdout:(85/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-09-19T03:54:22.831 INFO:teuthology.orchestra.run.smithi111.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 1.5 MB/s | 1.4 MB 00:00 2024-09-19T03:54:22.847 INFO:teuthology.orchestra.run.smithi019.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-09-19T03:54:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-19T03:54:22.939 INFO:teuthology.orchestra.run.smithi019.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-19T03:54:22.981 INFO:teuthology.orchestra.run.smithi019.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-19T03:54:23.023 INFO:teuthology.orchestra.run.smithi019.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-19T03:54:23.048 INFO:teuthology.orchestra.run.smithi019.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-09-19T03:54:23.090 INFO:teuthology.orchestra.run.smithi019.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-19T03:54:23.132 INFO:teuthology.orchestra.run.smithi019.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-19T03:54:23.216 INFO:teuthology.orchestra.run.smithi019.stdout:(94/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.2 MB/s | 5.1 MB 00:01 2024-09-19T03:54:23.258 INFO:teuthology.orchestra.run.smithi019.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-19T03:54:23.283 INFO:teuthology.orchestra.run.smithi019.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-09-19T03:54:23.317 INFO:teuthology.orchestra.run.smithi019.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-19T03:54:23.359 INFO:teuthology.orchestra.run.smithi019.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-19T03:54:23.365 INFO:teuthology.orchestra.run.smithi111.stdout:(155/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.1 MB/s | 16 MB 00:05 2024-09-19T03:54:23.417 INFO:teuthology.orchestra.run.smithi019.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-09-19T03:54:23.459 INFO:teuthology.orchestra.run.smithi019.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-09-19T03:54:23.485 INFO:teuthology.orchestra.run.smithi019.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-09-19T03:54:23.490 INFO:teuthology.orchestra.run.smithi111.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-09-19T03:54:23.495 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:54:23.495 INFO:teuthology.orchestra.run.smithi111.stdout:Total 14 MB/s | 226 MB 00:16 2024-09-19T03:54:23.510 INFO:teuthology.orchestra.run.smithi019.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-19T03:54:23.594 INFO:teuthology.orchestra.run.smithi019.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-09-19T03:54:23.642 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-19T03:54:23.642 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-19T03:54:23.652 INFO:teuthology.orchestra.run.smithi019.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T03:54:23.686 INFO:teuthology.orchestra.run.smithi019.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T03:54:23.711 INFO:teuthology.orchestra.run.smithi019.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-19T03:54:23.828 INFO:teuthology.orchestra.run.smithi019.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-19T03:54:23.853 INFO:teuthology.orchestra.run.smithi019.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T03:54:23.878 INFO:teuthology.orchestra.run.smithi019.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T03:54:23.904 INFO:teuthology.orchestra.run.smithi019.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T03:54:23.929 INFO:teuthology.orchestra.run.smithi019.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-19T03:54:23.963 INFO:teuthology.orchestra.run.smithi019.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-19T03:54:24.005 INFO:teuthology.orchestra.run.smithi019.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-19T03:54:24.030 INFO:teuthology.orchestra.run.smithi019.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.9 MB/s | 43 kB 00:00 2024-09-19T03:54:24.055 INFO:teuthology.orchestra.run.smithi019.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T03:54:24.081 INFO:teuthology.orchestra.run.smithi019.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-19T03:54:24.106 INFO:teuthology.orchestra.run.smithi019.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-19T03:54:24.131 INFO:teuthology.orchestra.run.smithi019.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-09-19T03:54:24.165 INFO:teuthology.orchestra.run.smithi019.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T03:54:24.190 INFO:teuthology.orchestra.run.smithi019.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T03:54:24.215 INFO:teuthology.orchestra.run.smithi019.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T03:54:24.249 INFO:teuthology.orchestra.run.smithi019.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T03:54:24.275 INFO:teuthology.orchestra.run.smithi019.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-19T03:54:24.300 INFO:teuthology.orchestra.run.smithi019.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-19T03:54:24.325 INFO:teuthology.orchestra.run.smithi019.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-19T03:54:24.353 INFO:teuthology.orchestra.run.smithi019.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-09-19T03:54:24.385 INFO:teuthology.orchestra.run.smithi019.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 581 kB/s | 20 kB 00:00 2024-09-19T03:54:24.435 INFO:teuthology.orchestra.run.smithi019.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.3 MB/s | 2.0 MB 00:00 2024-09-19T03:54:24.460 INFO:teuthology.orchestra.run.smithi019.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T03:54:24.486 INFO:teuthology.orchestra.run.smithi019.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T03:54:24.511 INFO:teuthology.orchestra.run.smithi019.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T03:54:24.553 INFO:teuthology.orchestra.run.smithi019.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-19T03:54:24.587 INFO:teuthology.orchestra.run.smithi019.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T03:54:24.612 INFO:teuthology.orchestra.run.smithi019.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-19T03:54:24.637 INFO:teuthology.orchestra.run.smithi019.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T03:54:24.662 INFO:teuthology.orchestra.run.smithi019.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-19T03:54:24.688 INFO:teuthology.orchestra.run.smithi019.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T03:54:24.713 INFO:teuthology.orchestra.run.smithi019.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T03:54:24.738 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-19T03:54:24.738 INFO:teuthology.orchestra.run.smithi019.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T03:54:24.765 INFO:teuthology.orchestra.run.smithi019.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-19T03:54:24.789 INFO:teuthology.orchestra.run.smithi019.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-19T03:54:24.814 INFO:teuthology.orchestra.run.smithi019.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T03:54:24.848 INFO:teuthology.orchestra.run.smithi019.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-19T03:54:24.873 INFO:teuthology.orchestra.run.smithi019.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-19T03:54:24.888 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-19T03:54:24.888 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-19T03:54:24.898 INFO:teuthology.orchestra.run.smithi019.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-19T03:54:24.923 INFO:teuthology.orchestra.run.smithi019.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-19T03:54:24.999 INFO:teuthology.orchestra.run.smithi019.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T03:54:25.316 INFO:teuthology.orchestra.run.smithi019.stdout:(148/156): librbd1-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-19T03:54:25.408 INFO:teuthology.orchestra.run.smithi019.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 7.1 MB/s | 3.4 MB 00:00 2024-09-19T03:54:26.150 INFO:teuthology.orchestra.run.smithi019.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.8 MB/s | 2.3 MB 00:00 2024-09-19T03:54:26.266 INFO:teuthology.orchestra.run.smithi019.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 352 kB/s | 41 kB 00:00 2024-09-19T03:54:26.325 INFO:teuthology.orchestra.run.smithi019.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 2.0 MB/s | 1.8 MB 00:00 2024-09-19T03:54:26.384 INFO:teuthology.orchestra.run.smithi019.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-19T03:54:26.609 INFO:teuthology.orchestra.run.smithi019.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-19T03:54:26.692 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-19T03:54:26.693 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-19T03:54:26.735 INFO:teuthology.orchestra.run.smithi019.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.3 MB/s | 1.4 MB 00:00 2024-09-19T03:54:28.326 INFO:teuthology.orchestra.run.smithi019.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.1 MB/s | 16 MB 00:05 2024-09-19T03:54:28.332 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:54:28.332 INFO:teuthology.orchestra.run.smithi019.stdout:Total 13 MB/s | 226 MB 00:17 2024-09-19T03:54:29.560 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-19T03:54:29.731 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-19T03:54:29.731 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-19T03:54:30.049 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-19T03:54:30.181 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T03:54:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T03:54:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T03:54:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T03:54:30.340 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T03:54:30.383 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T03:54:30.715 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T03:54:30.825 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:54:30.947 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:54:30.976 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:54:31.066 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:54:31.097 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T03:54:31.246 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T03:54:31.310 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T03:54:31.338 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:54:31.441 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:54:31.464 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T03:54:31.535 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T03:54:31.557 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T03:54:31.583 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T03:54:31.767 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T03:54:31.921 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T03:54:32.067 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T03:54:32.072 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-19T03:54:32.073 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-19T03:54:32.170 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:32.235 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:32.444 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:32.566 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T03:54:32.653 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T03:54:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T03:54:32.725 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T03:54:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T03:54:32.843 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T03:54:32.875 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T03:54:32.919 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T03:54:32.951 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T03:54:32.997 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T03:54:33.081 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T03:54:33.103 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-19T03:54:33.144 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T03:54:33.174 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T03:54:33.218 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T03:54:33.227 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T03:54:33.294 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T03:54:33.330 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T03:54:33.364 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T03:54:33.373 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T03:54:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T03:54:33.480 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T03:54:33.525 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T03:54:33.530 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T03:54:33.570 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T03:54:33.595 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T03:54:33.609 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T03:54:33.649 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T03:54:33.753 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T03:54:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T03:54:33.987 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T03:54:34.096 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:54:34.209 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:54:34.239 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:54:34.322 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:54:34.351 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T03:54:34.361 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T03:54:34.441 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T03:54:34.478 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T03:54:34.501 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T03:54:34.523 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T03:54:34.561 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T03:54:34.573 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T03:54:34.601 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:54:34.603 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T03:54:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:54:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T03:54:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T03:54:34.792 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T03:54:34.811 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T03:54:34.831 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T03:54:34.833 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T03:54:34.886 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T03:54:34.958 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T03:54:34.988 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T03:54:34.991 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T03:54:35.029 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T03:54:35.082 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T03:54:35.111 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T03:54:35.139 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T03:54:35.153 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T03:54:35.212 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T03:54:35.252 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T03:54:35.346 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T03:54:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T03:54:35.411 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:35.438 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T03:54:35.541 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T03:54:35.630 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:35.681 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T03:54:35.720 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T03:54:35.746 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T03:54:35.746 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T03:54:35.780 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T03:54:35.838 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T03:54:35.853 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T03:54:35.866 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T03:54:35.905 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T03:54:35.980 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T03:54:35.987 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T03:54:36.020 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T03:54:36.039 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T03:54:36.048 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T03:54:36.079 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T03:54:36.082 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T03:54:36.121 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T03:54:36.122 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T03:54:36.153 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T03:54:36.155 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T03:54:36.183 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T03:54:36.184 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T03:54:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T03:54:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T03:54:36.252 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T03:54:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T03:54:36.305 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T03:54:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T03:54:36.345 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T03:54:36.396 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T03:54:36.414 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T03:54:36.466 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T03:54:36.475 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T03:54:36.497 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T03:54:36.507 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T03:54:36.551 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T03:54:36.579 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T03:54:36.599 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T03:54:36.647 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T03:54:36.649 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T03:54:36.694 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T03:54:36.768 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T03:54:36.911 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T03:54:36.963 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T03:54:37.367 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T03:54:37.420 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T03:54:37.463 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T03:54:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T03:54:37.561 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T03:54:37.576 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T03:54:37.605 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T03:54:37.617 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T03:54:37.647 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T03:54:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T03:54:37.685 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T03:54:37.727 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T03:54:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T03:54:37.773 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T03:54:37.895 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T03:54:37.955 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T03:54:37.956 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T03:54:37.980 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T03:54:38.010 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T03:54:38.057 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T03:54:38.092 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T03:54:38.132 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T03:54:38.168 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T03:54:38.214 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T03:54:38.250 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T03:54:38.294 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T03:54:38.336 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T03:54:38.474 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T03:54:38.501 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-19T03:54:38.529 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T03:54:38.597 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T03:54:38.636 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T03:54:38.671 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T03:54:38.695 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T03:54:38.710 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T03:54:38.731 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T03:54:38.775 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T03:54:38.781 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T03:54:38.812 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T03:54:38.819 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T03:54:38.846 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T03:54:38.852 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T03:54:38.878 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T03:54:38.903 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T03:54:38.978 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T03:54:39.104 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T03:54:39.135 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T03:54:39.163 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T03:54:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T03:54:39.197 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T03:54:39.237 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T03:54:39.268 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T03:54:39.298 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T03:54:39.332 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T03:54:39.366 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T03:54:39.407 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T03:54:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:54:39.442 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T03:54:39.538 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T03:54:39.599 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T03:54:39.637 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T03:54:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:54:39.740 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T03:54:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:54:39.788 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T03:54:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:54:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T03:54:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T03:54:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T03:54:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T03:54:40.117 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T03:54:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:54:40.145 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T03:54:40.187 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T03:54:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:54:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T03:54:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T03:54:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T03:54:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T03:54:40.495 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T03:54:40.511 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T03:54:40.541 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T03:54:40.559 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T03:54:40.594 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T03:54:40.623 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T03:54:40.654 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T03:54:40.692 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T03:54:40.740 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T03:54:40.785 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T03:54:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T03:54:40.883 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T03:54:40.892 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T03:54:40.946 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T03:54:41.014 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T03:54:41.130 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T03:54:41.131 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T03:54:41.161 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T03:54:41.238 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T03:54:41.246 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T03:54:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T03:54:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T03:54:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T03:54:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T03:54:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T03:54:41.778 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T03:54:41.813 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T03:54:41.856 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T03:54:41.888 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T03:54:41.930 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T03:54:41.982 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T03:54:41.993 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T03:54:42.030 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T03:54:42.082 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T03:54:42.115 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T03:54:42.142 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T03:54:42.213 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T03:54:42.265 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T03:54:42.295 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T03:54:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T03:54:42.425 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T03:54:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T03:54:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T03:54:42.546 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T03:54:42.620 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T03:54:42.662 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T03:54:42.739 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T03:54:42.885 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T03:54:42.940 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T03:54:43.225 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T03:54:43.322 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T03:54:43.350 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T03:54:43.384 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T03:54:43.498 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T03:54:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T03:54:43.621 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T03:54:43.665 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T03:54:43.703 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T03:54:43.719 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T03:54:43.746 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T03:54:43.755 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T03:54:43.775 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:43.845 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T03:54:43.877 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T03:54:43.909 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T03:54:43.912 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T03:54:43.964 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T03:54:44.021 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T03:54:44.026 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:44.094 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T03:54:44.140 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T03:54:44.176 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T03:54:44.232 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T03:54:44.267 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T03:54:44.311 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T03:54:44.356 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T03:54:44.397 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T03:54:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T03:54:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T03:54:44.576 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:44.583 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T03:54:44.613 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T03:54:44.664 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T03:54:44.688 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T03:54:44.706 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T03:54:44.706 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T03:54:44.706 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T03:54:44.707 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:54:44.724 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T03:54:44.739 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T03:54:44.748 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T03:54:44.765 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T03:54:44.778 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T03:54:44.798 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T03:54:44.822 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T03:54:44.910 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T03:54:45.037 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T03:54:45.072 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T03:54:45.108 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T03:54:45.142 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T03:54:45.190 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T03:54:45.238 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T03:54:45.268 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T03:54:45.302 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T03:54:45.335 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T03:54:45.377 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T03:54:45.411 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T03:54:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T03:54:45.551 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T03:54:45.584 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T03:54:45.692 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T03:54:45.753 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T03:54:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T03:54:46.565 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T03:54:46.599 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T03:54:46.703 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T03:54:46.755 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T03:54:46.801 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T03:54:46.915 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T03:54:46.952 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T03:54:47.030 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:47.066 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:47.144 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T03:54:47.174 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T03:54:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T03:54:47.613 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:47.653 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T03:54:47.744 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T03:54:47.745 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T03:54:47.745 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T03:54:47.745 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:54:47.787 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T03:54:47.947 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T03:54:48.068 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T03:54:49.386 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T03:54:49.462 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T03:54:49.489 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T03:54:49.522 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T03:54:49.850 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T03:54:49.893 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T03:54:49.968 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T03:54:49.999 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T03:54:50.040 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T03:54:50.554 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T03:54:50.596 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T03:54:50.668 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T03:54:50.707 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T03:54:53.118 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:53.153 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T03:54:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T03:54:53.881 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T03:54:53.881 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T03:54:53.881 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T03:54:53.881 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:54:53.916 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T03:54:56.469 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T03:54:56.469 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-09-19T03:54:56.470 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-09-19T03:54:56.470 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-09-19T03:54:56.470 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-09-19T03:54:56.470 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-09-19T03:54:56.470 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-09-19T03:54:56.470 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-09-19T03:54:56.470 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:54:56.643 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T03:54:56.659 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T03:54:56.659 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:54:56.659 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T03:54:56.659 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:54:58.162 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T03:54:58.189 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T03:54:58.190 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:54:58.190 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T03:54:58.190 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T03:54:58.190 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T03:54:58.190 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:54:58.430 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T03:54:58.454 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T03:54:58.454 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:54:58.454 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T03:54:58.454 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T03:54:58.454 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T03:54:58.454 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:54:58.897 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T03:54:58.921 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T03:54:58.921 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:54:58.921 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T03:54:58.921 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T03:54:58.921 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T03:54:58.921 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:54:59.027 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T03:54:59.076 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T03:54:59.132 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T03:54:59.314 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T03:54:59.365 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T03:54:59.442 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T03:54:59.523 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T03:54:59.523 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-09-19T03:54:59.523 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-09-19T03:54:59.523 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-09-19T03:54:59.523 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-09-19T03:54:59.523 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-09-19T03:54:59.523 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-09-19T03:54:59.524 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-09-19T03:54:59.524 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:54:59.531 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T03:54:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T03:54:59.689 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T03:54:59.705 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T03:54:59.705 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:54:59.705 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T03:54:59.705 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:54:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T03:54:59.746 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T03:54:59.774 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T03:55:00.262 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T03:55:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T03:55:00.321 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T03:55:00.383 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T03:55:00.413 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T03:55:01.191 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T03:55:01.218 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T03:55:01.218 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:01.218 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T03:55:01.218 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T03:55:01.218 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T03:55:01.218 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:01.456 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T03:55:01.459 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T03:55:01.483 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T03:55:01.483 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:01.484 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T03:55:01.484 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T03:55:01.484 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T03:55:01.484 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:01.489 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T03:55:01.928 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T03:55:01.951 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T03:55:01.951 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:01.951 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T03:55:01.951 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T03:55:01.951 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T03:55:01.952 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:02.049 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T03:55:02.107 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T03:55:02.171 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T03:55:02.360 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T03:55:02.412 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T03:55:02.448 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T03:55:02.473 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T03:55:02.500 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T03:55:02.576 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T03:55:02.596 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T03:55:02.746 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T03:55:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T03:55:02.784 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T03:55:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T03:55:02.808 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T03:55:02.808 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:02.808 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T03:55:02.808 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T03:55:02.808 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T03:55:02.808 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:55:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T03:55:02.846 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T03:55:02.860 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T03:55:02.871 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T03:55:02.969 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T03:55:03.016 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T03:55:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T03:55:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T03:55:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T03:55:03.421 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T03:55:03.451 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T03:55:03.938 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T03:55:03.971 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T03:55:03.994 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T03:55:03.994 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:03.994 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T03:55:03.994 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T03:55:03.994 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T03:55:03.994 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:55:04.038 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T03:55:04.061 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T03:55:04.061 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:04.061 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T03:55:04.061 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:55:04.113 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T03:55:04.137 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T03:55:04.137 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:04.137 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T03:55:04.137 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T03:55:04.137 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T03:55:04.137 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:55:04.456 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T03:55:04.480 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T03:55:04.480 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:04.480 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T03:55:04.480 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T03:55:04.480 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T03:55:04.480 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:55:04.502 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T03:55:04.553 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T03:55:05.521 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T03:55:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T03:55:05.657 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T03:55:05.815 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T03:55:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T03:55:05.872 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T03:55:05.872 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:05.872 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T03:55:05.872 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T03:55:05.872 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T03:55:05.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:05.923 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T03:55:05.935 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T03:55:06.033 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T03:55:06.080 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T03:55:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T03:55:06.333 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-09-19T03:55:06.333 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-09-19T03:55:06.333 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-09-19T03:55:06.334 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-09-19T03:55:06.334 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-09-19T03:55:06.334 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-09-19T03:55:06.334 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-09-19T03:55:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T03:55:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T03:55:06.535 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:06.535 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T03:55:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:07.004 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T03:55:07.034 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T03:55:07.058 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T03:55:07.058 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:07.058 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T03:55:07.059 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T03:55:07.059 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T03:55:07.059 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:07.110 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T03:55:07.134 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T03:55:07.134 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:07.134 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T03:55:07.134 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:07.186 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T03:55:07.209 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T03:55:07.210 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:07.210 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T03:55:07.210 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T03:55:07.210 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T03:55:07.210 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:07.529 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T03:55:07.551 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T03:55:07.551 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:07.552 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T03:55:07.552 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T03:55:07.552 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T03:55:07.552 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:08.080 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T03:55:08.108 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T03:55:08.108 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:08.108 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T03:55:08.109 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T03:55:08.109 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T03:55:08.109 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:08.356 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T03:55:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T03:55:08.381 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:08.381 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T03:55:08.381 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T03:55:08.381 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T03:55:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T03:55:08.872 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T03:55:08.872 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:08.872 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T03:55:08.873 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T03:55:08.873 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T03:55:08.873 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:08.979 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T03:55:09.035 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T03:55:09.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T03:55:09.076 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T03:55:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T03:55:09.116 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T03:55:09.149 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T03:55:09.176 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T03:55:09.306 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T03:55:09.358 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T03:55:09.359 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T03:55:09.407 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T03:55:09.436 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T03:55:09.448 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T03:55:09.466 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T03:55:09.513 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T03:55:09.545 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T03:55:09.697 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T03:55:09.748 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T03:55:09.781 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T03:55:09.817 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T03:55:10.050 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:10.094 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:10.115 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:10.256 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T03:55:10.283 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T03:55:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T03:55:10.389 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T03:55:10.413 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T03:55:10.941 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T03:55:11.024 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T03:55:11.064 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T03:55:11.144 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T03:55:11.185 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T03:55:11.210 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T03:55:11.245 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T03:55:11.245 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T03:55:11.346 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T03:55:11.346 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T03:55:11.438 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T03:55:11.463 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T03:55:11.469 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:11.469 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:11.528 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T03:55:11.565 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T03:55:11.648 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:11.648 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T03:55:11.716 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T03:55:11.787 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T03:55:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T03:55:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T03:55:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T03:55:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T03:55:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T03:55:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T03:55:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T03:55:12.537 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T03:55:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T03:55:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T03:55:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T03:55:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T03:55:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T03:55:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T03:55:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T03:55:12.951 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:12.952 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T03:55:12.952 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T03:55:12.952 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T03:55:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T03:55:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T03:55:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T03:55:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T03:55:13.207 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:13.249 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:13.270 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T03:55:14.164 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T03:55:14.205 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T03:55:14.220 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T03:55:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T03:55:14.229 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:14.229 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T03:55:14.229 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T03:55:14.229 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T03:55:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:14.261 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T03:55:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T03:55:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T03:55:14.295 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:14.295 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T03:55:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:14.343 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T03:55:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T03:55:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T03:55:14.372 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:14.372 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T03:55:14.372 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T03:55:14.372 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T03:55:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:14.373 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T03:55:14.398 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T03:55:14.424 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T03:55:14.424 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T03:55:14.500 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T03:55:14.500 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T03:55:14.584 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T03:55:14.609 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T03:55:14.615 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:14.615 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T03:55:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T03:55:14.714 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T03:55:14.714 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T03:55:14.714 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T03:55:14.714 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T03:55:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:14.794 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:14.795 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T03:55:14.870 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T03:55:14.933 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T03:55:17.552 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T03:55:17.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T03:55:17.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T03:55:17.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T03:55:17.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T03:55:17.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T03:55:17.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T03:55:17.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T03:55:17.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T03:55:17.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T03:55:17.562 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T03:55:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T03:55:17.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T03:55:17.565 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T03:55:19.569 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T03:55:19.610 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T03:55:19.660 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T03:55:19.700 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T03:55:19.727 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T03:55:19.937 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T03:55:19.982 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T03:55:20.012 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T03:55:20.042 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T03:55:20.096 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T03:55:20.200 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:20.201 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.202 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.203 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T03:55:20.204 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.205 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T03:55:20.206 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T03:55:20.207 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T03:55:20.208 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T03:55:20.209 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T03:55:20.210 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T03:55:20.211 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T03:55:20.212 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T03:55:20.213 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T03:55:20.214 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T03:55:20.214 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:55:20.214 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-19T03:55:20.637 DEBUG:teuthology.parallel:result is None 2024-09-19T03:55:20.749 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:20.785 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:20.806 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T03:55:20.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T03:55:20.822 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T03:55:20.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T03:55:20.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T03:55:20.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T03:55:20.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T03:55:20.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T03:55:20.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T03:55:20.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T03:55:20.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T03:55:20.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T03:55:20.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T03:55:20.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T03:55:20.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T03:55:20.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T03:55:20.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T03:55:20.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T03:55:20.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T03:55:20.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T03:55:20.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T03:55:20.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T03:55:20.846 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T03:55:20.846 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T03:55:20.846 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T03:55:20.846 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T03:55:21.724 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T03:55:21.865 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T03:55:21.916 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T03:55:22.016 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T03:55:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T03:55:22.077 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T03:55:22.111 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T03:55:22.111 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T03:55:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T03:55:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T03:55:22.279 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T03:55:22.305 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T03:55:22.310 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:22.310 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:22.490 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T03:55:22.490 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T03:55:22.590 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T03:55:22.667 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:23.603 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:23.603 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:23.603 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-09-19T03:55:23.603 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-19T03:55:23.604 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.605 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.606 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T03:55:23.607 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.608 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T03:55:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T03:55:23.610 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T03:55:23.611 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T03:55:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T03:55:23.613 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T03:55:23.614 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T03:55:23.615 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:23.616 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T03:55:23.617 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T03:55:23.617 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T03:55:23.617 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T03:55:23.617 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T03:55:23.617 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T03:55:23.618 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T03:55:23.618 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:55:23.618 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-19T03:55:24.047 DEBUG:teuthology.parallel:result is None 2024-09-19T03:55:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T03:55:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T03:55:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T03:55:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T03:55:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T03:55:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T03:55:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T03:55:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T03:55:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T03:55:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T03:55:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T03:55:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T03:55:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T03:55:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T03:55:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T03:55:28.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T03:55:28.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T03:55:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T03:55:28.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T03:55:28.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T03:55:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T03:55:28.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T03:55:28.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T03:55:28.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T03:55:28.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T03:55:28.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T03:55:28.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T03:55:28.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T03:55:31.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T03:55:31.097 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:31.097 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-09-19T03:55:31.097 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T03:55:31.098 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T03:55:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T03:55:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T03:55:31.103 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T03:55:31.103 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T03:55:31.103 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T03:55:31.103 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T03:55:31.103 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T03:55:31.103 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T03:55:31.103 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T03:55:31.103 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T03:55:31.104 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T03:55:31.105 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T03:55:31.106 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T03:55:31.107 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T03:55:31.108 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T03:55:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T03:55:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T03:55:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T03:55:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T03:55:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T03:55:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T03:55:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T03:55:31.111 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:55:31.111 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-19T03:55:31.512 DEBUG:teuthology.parallel:result is None 2024-09-19T03:55:31.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:55:31.652 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T03:55:31.675 INFO:teuthology.orchestra.run.smithi019.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T03:55:31.676 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T03:55:31.676 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T03:55:31.678 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:55:31.811 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T03:55:31.837 INFO:teuthology.orchestra.run.smithi111.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T03:55:31.838 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T03:55:31.838 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T03:55:31.839 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:55:31.974 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T03:55:32.001 INFO:teuthology.orchestra.run.smithi196.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T03:55:32.002 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T03:55:32.002 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T03:55:32.003 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T03:55:32.004 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:55:32.004 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T03:55:32.036 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:55:32.036 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T03:55:32.069 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:55:32.069 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T03:55:32.099 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T03:55:32.099 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:55:32.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T03:55:32.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T03:55:32.196 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:55:32.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T03:55:32.226 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T03:55:32.295 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:55:32.295 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T03:55:32.325 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T03:55:32.395 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T03:55:32.395 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:55:32.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T03:55:32.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T03:55:32.491 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:55:32.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T03:55:32.521 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T03:55:32.591 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:55:32.591 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T03:55:32.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T03:55:32.683 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T03:55:32.683 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:55:32.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T03:55:32.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T03:55:32.779 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:55:32.780 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T03:55:32.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T03:55:32.879 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:55:32.879 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T03:55:32.905 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T03:55:32.970 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T03:55:33.243 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2}} 2024-09-19T03:55:33.243 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:55:33.244 INFO:tasks.cephadm:Cluster fsid is 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:55:33.244 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T03:55:33.244 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.111', 'mon.c': '172.21.15.196'} 2024-09-19T03:55:33.245 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-09-19T03:55:33.245 INFO:tasks.cephadm:First mgr is x 2024-09-19T03:55:33.245 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T03:55:33.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-09-19T03:55:33.276 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2024-09-19T03:55:33.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-09-19T03:55:33.334 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T03:55:33.335 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:55:33.472 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T03:55:33.605 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:55:33.607 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T03:55:33.607 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T03:55:33.607 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.022 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 03:55 /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.022 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.420 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 03:55 /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.420 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.800 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 03:55 /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.800 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.820 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.840 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T03:55:34.875 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T03:55:34.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T03:55:34.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T03:55:34.882 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T03:55:36.423 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T03:55:36.530 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T03:55:36.549 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T03:56:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-19T03:56:05.598 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T03:56:05.598 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T03:56:05.598 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2024-09-19T03:56:05.598 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T03:56:05.598 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-19T03:56:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-19T03:56:33.295 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T03:56:33.295 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T03:56:33.295 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T03:56:33.295 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-09-19T03:56:33.295 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T03:56:33.295 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T03:56:33.295 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T03:56:33.377 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T03:56:33.377 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T03:56:33.377 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T03:56:33.377 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-09-19T03:56:33.377 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T03:56:33.378 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-19T03:56:33.378 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T03:56:33.402 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-09-19T03:56:33.432 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2024-09-19T03:56:33.467 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-09-19T03:56:33.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-09-19T03:56:33.524 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2024-09-19T03:56:33.555 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-09-19T03:56:33.580 INFO:tasks.cephadm:Writing seed config... 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T03:56:33.584 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T03:56:33.585 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T03:56:33.586 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T03:56:33.587 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T03:56:33.588 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:56:33.588 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T03:56:33.605 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0518de98-763b-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T03:56:33.605 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T03:56:33.648 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T03:56:33.691 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T03:56:33.691 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 0518de98-763b-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T03:56:33.901 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-19T03:56:33.901 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '0518de98-763b-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-09-19T03:56:33.901 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T03:56:33.904 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-09-19T03:56:33.926 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 5.2.2 2024-09-19T03:56:33.926 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-09-19T03:56:33.926 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-09-19T03:56:33.934 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T03:56:33.934 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T03:56:33.940 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T03:56:33.941 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-19T03:56:33.947 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-09-19T03:56:33.954 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-09-19T03:56:33.954 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-09-19T03:56:33.954 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-09-19T03:56:33.977 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 5.2.2 2024-09-19T03:56:33.978 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T03:56:33.978 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-09-19T03:56:33.978 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-09-19T03:56:33.984 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T03:56:33.985 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T03:56:33.991 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T03:56:33.991 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-19T03:56:33.998 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-09-19T03:56:34.004 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-09-19T03:56:34.005 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-09-19T03:56:34.005 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-09-19T03:56:34.005 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:34.005 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 139814623945728 on /run/cephadm/0518de98-763b-11ef-bceb-c7b262605968.lock 2024-09-19T03:56:34.005 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 139814623945728 acquired on /run/cephadm/0518de98-763b-11ef-bceb-c7b262605968.lock 2024-09-19T03:56:34.005 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-09-19T03:56:34.006 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-09-19T03:56:34.006 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-09-19T03:56:34.009 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-09-19T03:56:34.009 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-09-19T03:56:34.012 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T03:56:34.012 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T03:56:34.012 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-09-19T03:56:34.015 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T03:56:34.015 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T03:56:34.015 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T03:56:34.015 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T03:56:34.015 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link 2024-09-19T03:56:34.015 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T03:56:34.016 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-09-19T03:56:34.016 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-09-19T03:56:34.016 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T03:56:34.016 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T03:56:34.017 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T03:56:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T03:56:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T03:56:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T03:56:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T03:56:34.518 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T03:56:34.518 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T03:56:34.518 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T03:56:35.444 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T03:56:35.444 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T03:56:35.444 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T03:56:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-09-19T03:56:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-09-19T03:56:37.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQD0oOtmZL0nLxAAQWlSgh10Oq3ULt9Mh4czEg== 2024-09-19T03:56:38.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQD1oOtmKPAZLBAAxw4+Oq2k7h7zUCxgV5t21Q== 2024-09-19T03:56:39.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQD2oOtmA8NdJBAAepTOKdSlFGRWh0PIeCo8qw== 2024-09-19T03:56:39.236 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T03:56:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:56:40.158 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-09-19T03:56:41.128 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-09-19T03:56:41.363 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T03:56:41.517 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T03:56:41.716 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0518de98-763b-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-0518de98-763b-11ef-bceb-c7b262605968.target. 2024-09-19T03:56:41.716 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0518de98-763b-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-0518de98-763b-11ef-bceb-c7b262605968.target. 2024-09-19T03:56:41.949 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a 2024-09-19T03:56:41.949 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T03:56:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0518de98-763b-11ef-bceb-c7b262605968.target.wants/ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0518de98-763b-11ef-bceb-c7b262605968@.service. 2024-09-19T03:56:42.655 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T03:56:42.655 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-09-19T03:56:42.663 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T03:56:42.663 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-19T03:56:42.663 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-09-19T03:56:42.663 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T03:56:42.663 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-09-19T03:56:42.663 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-09-19T03:56:42.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:42 smithi019 systemd[1]: Started Ceph mon.a for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T03:56:43.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T03:56:43.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: monmap epoch 1 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: last_changed 2024-09-19T03:56:39.538905+0000 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: created 2024-09-19T03:56:39.538905+0000 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: min_mon_release 19 (squid) 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: election_strategy: 1 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: fsmap 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: mgrmap e1: no daemons active 2024-09-19T03:56:43.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:43 smithi019 ceph-mon[30561]: from='client.? 172.21.15.19:0/2046774871' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.5613s) 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T03:56:43.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.071+0000 7f4f510fe640 1 Processor -- start 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.071+0000 7f4f510fe640 1 -- start start 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.071+0000 7f4f510fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c10a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.071+0000 7f4f510fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c07bed0 con 0x7f4f4c1080f0 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.072+0000 7f4f4ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c10a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.072+0000 7f4f4ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c10a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46098/0 (socket says 172.21.15.19:46098) 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.072+0000 7f4f4ad9d640 1 -- 172.21.15.19:0/2119382682 learned_addr learned my addr 172.21.15.19:0/2119382682 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.072+0000 7f4f4ad9d640 1 -- 172.21.15.19:0/2119382682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c10aa20 con 0x7f4f4c1080f0 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.074+0000 7f4f4ad9d640 1 --2- 172.21.15.19:0/2119382682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c10a4e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009820 tx=0x7f4f3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2313cc59c54b85a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.074+0000 7f4f4a59c640 1 -- 172.21.15.19:0/2119382682 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f3c037070 con 0x7f4f4c1080f0 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.074+0000 7f4f4a59c640 1 -- 172.21.15.19:0/2119382682 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f3c032440 con 0x7f4f4c1080f0 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.074+0000 7f4f4a59c640 1 -- 172.21.15.19:0/2119382682 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f3c032740 con 0x7f4f4c1080f0 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.075+0000 7f4f510fe640 1 -- 172.21.15.19:0/2119382682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 msgr2=0x7f4f4c10a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.075+0000 7f4f510fe640 1 --2- 172.21.15.19:0/2119382682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c10a4e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009820 tx=0x7f4f3c02a270 comp rx=0 tx=0).stop 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.075+0000 7f4f510fe640 1 -- 172.21.15.19:0/2119382682 shutdown_connections 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.075+0000 7f4f510fe640 1 --2- 172.21.15.19:0/2119382682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c10a4e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.075+0000 7f4f510fe640 1 -- 172.21.15.19:0/2119382682 >> 172.21.15.19:0/2119382682 conn(0x7f4f4c101d10 msgr2=0x7f4f4c104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:43.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.075+0000 7f4f510fe640 1 -- 172.21.15.19:0/2119382682 shutdown_connections 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.075+0000 7f4f510fe640 1 -- 172.21.15.19:0/2119382682 wait complete. 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.076+0000 7f4f510fe640 1 Processor -- start 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.076+0000 7f4f510fe640 1 -- start start 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.076+0000 7f4f510fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c107970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.076+0000 7f4f510fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c10af60 con 0x7f4f4c1080f0 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.076+0000 7f4f4ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.076+0000 7f4f4ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46108/0 (socket says 172.21.15.19:46108) 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.076+0000 7f4f4ad9d640 1 -- 172.21.15.19:0/2046774871 learned_addr learned my addr 172.21.15.19:0/2046774871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.077+0000 7f4f4ad9d640 1 -- 172.21.15.19:0/2046774871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c107eb0 con 0x7f4f4c1080f0 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.077+0000 7f4f4ad9d640 1 --2- 172.21.15.19:0/2046774871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c107970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0090b0 tx=0x7f4f3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.077+0000 7f4f48d99640 1 -- 172.21.15.19:0/2046774871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f3c041070 con 0x7f4f4c1080f0 2024-09-19T03:56:43.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.077+0000 7f4f48d99640 1 -- 172.21.15.19:0/2046774871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f3c03c7a0 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.077+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c1060a0 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.077+0000 7f4f48d99640 1 -- 172.21.15.19:0/2046774871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f3c037040 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.077+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c1065e0 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.078+0000 7f4f48d99640 1 -- 172.21.15.19:0/2046774871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4f3c044070 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.078+0000 7f4f48d99640 1 -- 172.21.15.19:0/2046774871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4f3c03d3f0 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.078+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c108550 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.081+0000 7f4f48d99640 1 -- 172.21.15.19:0/2046774871 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4f3c046450 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.155+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4f4c1947b0 con 0x7f4f4c1080f0 2024-09-19T03:56:43.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.156+0000 7f4f48d99640 1 -- 172.21.15.19:0/2046774871 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+315 (secure 0 0 0) 0x7f4f3c046630 con 0x7f4f4c1080f0 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.157+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 msgr2=0x7f4f4c107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.157+0000 7f4f510fe640 1 --2- 172.21.15.19:0/2046774871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c107970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0090b0 tx=0x7f4f3c0046c0 comp rx=0 tx=0).stop 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.157+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 shutdown_connections 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.157+0000 7f4f510fe640 1 --2- 172.21.15.19:0/2046774871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f4c1080f0 0x7f4f4c107970 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.157+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 >> 172.21.15.19:0/2046774871 conn(0x7f4f4c101d10 msgr2=0x7f4f4c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.157+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 shutdown_connections 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:43.157+0000 7f4f510fe640 1 -- 172.21.15.19:0/2046774871 wait complete. 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-09-19T03:56:43.768 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T03:56:44.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T03:56:44.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.193+0000 7f34f78db640 1 Processor -- start 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.193+0000 7f34f78db640 1 -- start start 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.193+0000 7f34f78db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f0107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.193+0000 7f34f78db640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34f01080f0 con 0x7f34f0107720 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.194+0000 7f34f5677640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f0107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:44.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.194+0000 7f34f5677640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f0107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46110/0 (socket says 172.21.15.19:46110) 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.194+0000 7f34f5677640 1 -- 172.21.15.19:0/360245019 learned_addr learned my addr 172.21.15.19:0/360245019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.194+0000 7f34f5677640 1 -- 172.21.15.19:0/360245019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f0108920 con 0x7f34f0107720 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.195+0000 7f34f5677640 1 --2- 172.21.15.19:0/360245019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f0107b20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009820 tx=0x7f34ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efcdfdc7ba73ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.195+0000 7f34e7fff640 1 -- 172.21.15.19:0/360245019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34ec037070 con 0x7f34f0107720 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.195+0000 7f34e7fff640 1 -- 172.21.15.19:0/360245019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec032440 con 0x7f34f0107720 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.195+0000 7f34e7fff640 1 -- 172.21.15.19:0/360245019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34ec032740 con 0x7f34f0107720 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 -- 172.21.15.19:0/360245019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 msgr2=0x7f34f0107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 --2- 172.21.15.19:0/360245019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f0107b20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009820 tx=0x7f34ec02a270 comp rx=0 tx=0).stop 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 -- 172.21.15.19:0/360245019 shutdown_connections 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 --2- 172.21.15.19:0/360245019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f0107b20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:44.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 -- 172.21.15.19:0/360245019 >> 172.21.15.19:0/360245019 conn(0x7f34f0102ed0 msgr2=0x7f34f01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 -- 172.21.15.19:0/360245019 shutdown_connections 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 -- 172.21.15.19:0/360245019 wait complete. 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 Processor -- start 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.196+0000 7f34f78db640 1 -- start start 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f78db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f78db640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34f0108e50 con 0x7f34f0107720 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f5677640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f5677640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46126/0 (socket says 172.21.15.19:46126) 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f5677640 1 -- 172.21.15.19:0/1825796970 learned_addr learned my addr 172.21.15.19:0/1825796970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f5677640 1 -- 172.21.15.19:0/1825796970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f019b3b0 con 0x7f34f0107720 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f5677640 1 --2- 172.21.15.19:0/1825796970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f019ae70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0090b0 tx=0x7f34ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34e67fc640 1 -- 172.21.15.19:0/1825796970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34ec041070 con 0x7f34f0107720 2024-09-19T03:56:44.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34e67fc640 1 -- 172.21.15.19:0/1825796970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec03c7a0 con 0x7f34f0107720 2024-09-19T03:56:44.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f019b640 con 0x7f34f0107720 2024-09-19T03:56:44.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34e67fc640 1 -- 172.21.15.19:0/1825796970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34ec037040 con 0x7f34f0107720 2024-09-19T03:56:44.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.197+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f019e330 con 0x7f34f0107720 2024-09-19T03:56:44.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.198+0000 7f34e67fc640 1 -- 172.21.15.19:0/1825796970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f34ec044070 con 0x7f34f0107720 2024-09-19T03:56:44.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.198+0000 7f34e67fc640 1 -- 172.21.15.19:0/1825796970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f34ec03d3f0 con 0x7f34f0107720 2024-09-19T03:56:44.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.198+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f01094a0 con 0x7f34f0107720 2024-09-19T03:56:44.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.201+0000 7f34e67fc640 1 -- 172.21.15.19:0/1825796970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f34ec046450 con 0x7f34f0107720 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.260+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f34f002d900 con 0x7f34f0107720 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.263+0000 7f34e67fc640 1 -- 172.21.15.19:0/1825796970 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f34ec046630 con 0x7f34f0107720 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.264+0000 7f34e67fc640 1 -- 172.21.15.19:0/1825796970 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f34ec056070 con 0x7f34f0107720 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.265+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 msgr2=0x7f34f019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.265+0000 7f34f78db640 1 --2- 172.21.15.19:0/1825796970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f019ae70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0090b0 tx=0x7f34ec0046c0 comp rx=0 tx=0).stop 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.265+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 shutdown_connections 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.265+0000 7f34f78db640 1 --2- 172.21.15.19:0/1825796970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0107720 0x7f34f019ae70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.265+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 >> 172.21.15.19:0/1825796970 conn(0x7f34f0102ed0 msgr2=0x7f34f018f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.265+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 shutdown_connections 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:44.265+0000 7f34f78db640 1 -- 172.21.15.19:0/1825796970 wait complete. 2024-09-19T03:56:44.818 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-09-19T03:56:44.929 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:44 smithi019 ceph-mon[30561]: from='client.? 172.21.15.19:0/1825796970' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T03:56:44.929 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:44 smithi019 ceph-mon[30561]: from='client.? 172.21.15.19:0/1825796970' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T03:56:45.893 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:45 smithi019 ceph-mon[30561]: from='client.? 172.21.15.19:0/4062843437' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:56:45.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.219+0000 7fd9d8019640 1 Processor -- start 2024-09-19T03:56:45.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.219+0000 7fd9d8019640 1 -- start start 2024-09-19T03:56:45.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.219+0000 7fd9d8019640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d0107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:45.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.219+0000 7fd9d8019640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d01080f0 con 0x7fd9d0107720 2024-09-19T03:56:45.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.219+0000 7fd9d5db5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d0107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:45.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.219+0000 7fd9d5db5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d0107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46136/0 (socket says 172.21.15.19:46136) 2024-09-19T03:56:45.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.219+0000 7fd9d5db5640 1 -- 172.21.15.19:0/1886627135 learned_addr learned my addr 172.21.15.19:0/1886627135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.220+0000 7fd9d5db5640 1 -- 172.21.15.19:0/1886627135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d0108920 con 0x7fd9d0107720 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.220+0000 7fd9d5db5640 1 --2- 172.21.15.19:0/1886627135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d0107b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009820 tx=0x7fd9cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df56115c99c71176 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d4db3640 1 -- 172.21.15.19:0/1886627135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9cc037070 con 0x7fd9d0107720 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d4db3640 1 -- 172.21.15.19:0/1886627135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd9cc032900 con 0x7fd9d0107720 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d4db3640 1 -- 172.21.15.19:0/1886627135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9cc030850 con 0x7fd9d0107720 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d8019640 1 -- 172.21.15.19:0/1886627135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 msgr2=0x7fd9d0107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d8019640 1 --2- 172.21.15.19:0/1886627135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d0107b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009820 tx=0x7fd9cc02a270 comp rx=0 tx=0).stop 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d8019640 1 -- 172.21.15.19:0/1886627135 shutdown_connections 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d8019640 1 --2- 172.21.15.19:0/1886627135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d0107b20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d8019640 1 -- 172.21.15.19:0/1886627135 >> 172.21.15.19:0/1886627135 conn(0x7fd9d0102ed0 msgr2=0x7fd9d01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:45.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d8019640 1 -- 172.21.15.19:0/1886627135 shutdown_connections 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.221+0000 7fd9d8019640 1 -- 172.21.15.19:0/1886627135 wait complete. 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.222+0000 7fd9d8019640 1 Processor -- start 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.222+0000 7fd9d8019640 1 -- start start 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.222+0000 7fd9d8019640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d007db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.222+0000 7fd9d8019640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d010bcd0 con 0x7fd9d0107720 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.222+0000 7fd9d5db5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d007db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.222+0000 7fd9d5db5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d007db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46144/0 (socket says 172.21.15.19:46144) 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.222+0000 7fd9d5db5640 1 -- 172.21.15.19:0/4062843437 learned_addr learned my addr 172.21.15.19:0/4062843437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.222+0000 7fd9d5db5640 1 -- 172.21.15.19:0/4062843437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d007e0d0 con 0x7fd9d0107720 2024-09-19T03:56:45.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.223+0000 7fd9d5db5640 1 --2- 172.21.15.19:0/4062843437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d007db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0097f0 tx=0x7fd9cc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.223+0000 7fd9c6ffd640 1 -- 172.21.15.19:0/4062843437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9cc042070 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.223+0000 7fd9c6ffd640 1 -- 172.21.15.19:0/4062843437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd9cc032ca0 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.223+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d007c2c0 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.223+0000 7fd9c6ffd640 1 -- 172.21.15.19:0/4062843437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9cc037040 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.223+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d007c800 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.223+0000 7fd9c6ffd640 1 -- 172.21.15.19:0/4062843437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd9cc045070 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.223+0000 7fd9c6ffd640 1 -- 172.21.15.19:0/4062843437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd9cc047a30 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.224+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d0107ba0 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.226+0000 7fd9c6ffd640 1 -- 172.21.15.19:0/4062843437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd9cc03da60 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.286+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd9d007cb00 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.286+0000 7fd9c6ffd640 1 -- 172.21.15.19:0/4062843437 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fd9cc03d430 con 0x7fd9d0107720 2024-09-19T03:56:45.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.287+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 msgr2=0x7fd9d007db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:45.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.287+0000 7fd9d8019640 1 --2- 172.21.15.19:0/4062843437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d007db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0097f0 tx=0x7fd9cc030ec0 comp rx=0 tx=0).stop 2024-09-19T03:56:45.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.287+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 shutdown_connections 2024-09-19T03:56:45.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.287+0000 7fd9d8019640 1 --2- 172.21.15.19:0/4062843437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9d0107720 0x7fd9d007db90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:45.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.287+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 >> 172.21.15.19:0/4062843437 conn(0x7fd9d0102ed0 msgr2=0x7fd9d0103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:45.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.287+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 shutdown_connections 2024-09-19T03:56:45.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:45.287+0000 7fd9d8019640 1 -- 172.21.15.19:0/4062843437 wait complete. 2024-09-19T03:56:45.905 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-09-19T03:56:46.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:45 smithi019 systemd[1]: Stopping Ceph mon.a for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T03:56:46.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:46 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a[30554]: 2024-09-19T03:56:46.179+0000 7fa8da5e5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T03:56:46.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:46 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a[30554]: 2024-09-19T03:56:46.179+0000 7fa8da5e5640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T03:56:46.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:46 smithi019 podman[30813]: 2024-09-19 03:56:46.282149424 +0000 UTC m=+0.211629440 container died e7a110c4d1701e110bc8b91357c5f30826f831fa4eeac94e029b5fe3c4a2b8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T03:56:46.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:46 smithi019 podman[30813]: 2024-09-19 03:56:46.612475948 +0000 UTC m=+0.541955968 container cleanup e7a110c4d1701e110bc8b91357c5f30826f831fa4eeac94e029b5fe3c4a2b8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T03:56:46.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:46 smithi019 bash[30813]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a 2024-09-19T03:56:47.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:46 smithi019 podman[30824]: 2024-09-19 03:56:46.796681099 +0000 UTC m=+0.509058288 container remove e7a110c4d1701e110bc8b91357c5f30826f831fa4eeac94e029b5fe3c4a2b8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T03:56:47.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T03:56:47.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 systemd[1]: Stopped Ceph mon.a for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T03:56:47.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 systemd[1]: Starting Ceph mon.a for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T03:56:47.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 podman[30919]: 2024-09-19 03:56:47.247373784 +0000 UTC m=+0.103494825 container create 4593a6e9f16a8508881d3133c95900c31aa771ab110d511be740d706420259e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T03:56:47.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 podman[30919]: 2024-09-19 03:56:47.159745677 +0000 UTC m=+0.015866728 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:56:47.546 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T03:56:47.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 podman[30919]: 2024-09-19 03:56:47.466512529 +0000 UTC m=+0.322633570 container init 4593a6e9f16a8508881d3133c95900c31aa771ab110d511be740d706420259e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T03:56:47.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 podman[30919]: 2024-09-19 03:56:47.469743945 +0000 UTC m=+0.325864989 container start 4593a6e9f16a8508881d3133c95900c31aa771ab110d511be740d706420259e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T03:56:47.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T03:56:47.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T03:56:47.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: pidfile_write: ignore empty --pid-file 2024-09-19T03:56:47.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: load: jerasure load: lrc 2024-09-19T03:56:47.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: RocksDB version: 7.9.2 2024-09-19T03:56:47.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Git sha 0 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: DB SUMMARY 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: DB Session ID: QGOOAGI8K626C8VJWOZY 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: CURRENT file: CURRENT 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105974 ; 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.error_if_exists: 0 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.create_if_missing: 0 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.paranoid_checks: 1 2024-09-19T03:56:47.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T03:56:47.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T03:56:47.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T03:56:47.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.env: 0x55d1dd32b940 2024-09-19T03:56:47.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T03:56:47.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.info_log: 0x55d1dde736e0 2024-09-19T03:56:47.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T03:56:47.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.statistics: (nil) 2024-09-19T03:56:47.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.use_fsync: 0 2024-09-19T03:56:47.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_log_file_size: 0 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.allow_fallocate: 1 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.use_direct_reads: 0 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T03:56:47.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.db_log_dir: 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.wal_dir: 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.write_buffer_manager: 0x55d1dde82dc0 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T03:56:47.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.unordered_write: 0 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T03:56:47.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.row_cache: None 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.wal_filter: None 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.two_write_queues: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.wal_compression: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.atomic_flush: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.log_readahead_size: 0 2024-09-19T03:56:47.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_background_jobs: 2 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_background_compactions: -1 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_subcompactions: 1 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T03:56:47.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_open_files: -1 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_background_flushes: -1 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Compression algorithms supported: 2024-09-19T03:56:47.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: kZSTD supported: 0 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: kXpressCompression supported: 0 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: kBZip2Compression supported: 0 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: kLZ4Compression supported: 1 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: kZlibCompression supported: 1 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: kSnappyCompression supported: 1 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T03:56:47.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.merge_operator: 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_filter: None 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d1dde73320) 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-09-19T03:56:47.679 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x55d1dde671f0 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-09-19T03:56:47.680 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-09-19T03:56:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression: NoCompression 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.num_levels: 7 2024-09-19T03:56:47.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T03:56:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T03:56:47.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T03:56:47.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T03:56:47.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.inplace_update_support: 0 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.bloom_locality: 0 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.max_successive_merges: 0 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.ttl: 2592000 2024-09-19T03:56:47.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.enable_blob_files: false 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.min_blob_size: 0 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T03:56:47.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7eeecfcc-e804-49ce-b802-2c112ed87f32 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718207496757, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718207497858, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100989, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98874, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14453, "raw_average_key_size": 50, "raw_value_size": 91046, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726718207, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7eeecfcc-e804-49ce-b802-2c112ed87f32", "db_session_id": "QGOOAGI8K626C8VJWOZY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718207497956, "job": 1, "event": "recovery_finished"} 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d1dde94e00 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: DB pointer 0x55d1ddf90000 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T03:56:47.689 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 100.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T03:56:47.690 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 100.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T03:56:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 29.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 29.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x55d1dde671f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-19T03:56:47.692 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a@-1(???) e1 preinit fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a@-1(???).mds e1 new map 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a@-1(???).mds e1 print_map 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout: btime 2024-09-19T03:56:42:596148+0000 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T03:56:47.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 bash[30919]: 4593a6e9f16a8508881d3133c95900c31aa771ab110d511be740d706420259e5 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 systemd[1]: Started Ceph mon.a for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: monmap epoch 1 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: last_changed 2024-09-19T03:56:39.538905+0000 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: created 2024-09-19T03:56:39.538905+0000 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: min_mon_release 19 (squid) 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: election_strategy: 1 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-19T03:56:47.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: fsmap 2024-09-19T03:56:47.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T03:56:47.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:47 smithi019 ceph-mon[30938]: mgrmap e1: no daemons active 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.053+0000 7f0c5aa70640 1 Processor -- start 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.054+0000 7f0c5aa70640 1 -- start start 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.054+0000 7f0c5aa70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c54106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.054+0000 7f0c5aa70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c54106f50 con 0x7f0c54106580 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.054+0000 7f0c5880c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c54106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.054+0000 7f0c5880c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c54106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46150/0 (socket says 172.21.15.19:46150) 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.054+0000 7f0c5880c640 1 -- 172.21.15.19:0/2143262599 learned_addr learned my addr 172.21.15.19:0/2143262599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.054+0000 7f0c5880c640 1 -- 172.21.15.19:0/2143262599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c54107780 con 0x7f0c54106580 2024-09-19T03:56:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.056+0000 7f0c5880c640 1 --2- 172.21.15.19:0/2143262599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c54106980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009820 tx=0x7f0c5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e3da8054ada953 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.056+0000 7f0c4b7fe640 1 -- 172.21.15.19:0/2143262599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c50037070 con 0x7f0c54106580 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.056+0000 7f0c4b7fe640 1 -- 172.21.15.19:0/2143262599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0c50032900 con 0x7f0c54106580 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.056+0000 7f0c4b7fe640 1 -- 172.21.15.19:0/2143262599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c50030850 con 0x7f0c54106580 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.057+0000 7f0c5aa70640 1 -- 172.21.15.19:0/2143262599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 msgr2=0x7f0c54106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.057+0000 7f0c5aa70640 1 --2- 172.21.15.19:0/2143262599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c54106980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009820 tx=0x7f0c5002a270 comp rx=0 tx=0).stop 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.057+0000 7f0c5aa70640 1 -- 172.21.15.19:0/2143262599 shutdown_connections 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.057+0000 7f0c5aa70640 1 --2- 172.21.15.19:0/2143262599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c54106980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.057+0000 7f0c5aa70640 1 -- 172.21.15.19:0/2143262599 >> 172.21.15.19:0/2143262599 conn(0x7f0c54101d10 msgr2=0x7f0c54104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.057+0000 7f0c5aa70640 1 -- 172.21.15.19:0/2143262599 shutdown_connections 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.057+0000 7f0c5aa70640 1 -- 172.21.15.19:0/2143262599 wait complete. 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5aa70640 1 Processor -- start 2024-09-19T03:56:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5aa70640 1 -- start start 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5aa70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c541917b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5aa70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5410ab30 con 0x7f0c54106580 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5880c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c541917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5880c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c541917b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46160/0 (socket says 172.21.15.19:46160) 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5880c640 1 -- 172.21.15.19:0/236021518 learned_addr learned my addr 172.21.15.19:0/236021518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5880c640 1 -- 172.21.15.19:0/236021518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c54191cf0 con 0x7f0c54106580 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.058+0000 7f0c5880c640 1 --2- 172.21.15.19:0/236021518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c541917b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0c500097f0 tx=0x7f0c50030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.059+0000 7f0c49ffb640 1 -- 172.21.15.19:0/236021518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c50037040 con 0x7f0c54106580 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.059+0000 7f0c49ffb640 1 -- 172.21.15.19:0/236021518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0c50040070 con 0x7f0c54106580 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.059+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5418fee0 con 0x7f0c54106580 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.059+0000 7f0c49ffb640 1 -- 172.21.15.19:0/236021518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c5003a390 con 0x7f0c54106580 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.059+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c541903c0 con 0x7f0c54106580 2024-09-19T03:56:48.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.059+0000 7f0c49ffb640 1 -- 172.21.15.19:0/236021518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0c5003a630 con 0x7f0c54106580 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.059+0000 7f0c49ffb640 1 -- 172.21.15.19:0/236021518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0c500459a0 con 0x7f0c54106580 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.060+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c5410acb0 con 0x7f0c54106580 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.062+0000 7f0c49ffb640 1 -- 172.21.15.19:0/236021518 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0c50045460 con 0x7f0c54106580 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.122+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f0c54106a00 con 0x7f0c54106580 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.124+0000 7f0c49ffb640 1 -- 172.21.15.19:0/236021518 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f0c5003b430 con 0x7f0c54106580 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.125+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 msgr2=0x7f0c541917b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.125+0000 7f0c5aa70640 1 --2- 172.21.15.19:0/236021518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c541917b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0c500097f0 tx=0x7f0c50030d80 comp rx=0 tx=0).stop 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.125+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 shutdown_connections 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.125+0000 7f0c5aa70640 1 --2- 172.21.15.19:0/236021518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c54106580 0x7f0c541917b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.125+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 >> 172.21.15.19:0/236021518 conn(0x7f0c54101d10 msgr2=0x7f0c54102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.125+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 shutdown_connections 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:48.125+0000 7f0c5aa70640 1 -- 172.21.15.19:0/236021518 wait complete. 2024-09-19T03:56:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T03:56:48.732 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T03:56:48.733 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-09-19T03:56:48.733 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T03:56:48.733 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T03:56:48.997 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x 2024-09-19T03:56:48.997 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T03:56:49.165 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0518de98-763b-11ef-bceb-c7b262605968.target.wants/ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0518de98-763b-11ef-bceb-c7b262605968@.service. 2024-09-19T03:56:49.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:49 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/236021518' entity='client.admin' 2024-09-19T03:56:49.393 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 systemd[1]: Starting Ceph mgr.x for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T03:56:49.671 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 podman[31149]: 2024-09-19 03:56:49.392613932 +0000 UTC m=+0.110684080 container create ffebeb9c157efaf2b4c8966bb0752a35dbe8e9dcd503975cdc838f92f57d806f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T03:56:49.671 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 podman[31149]: 2024-09-19 03:56:49.298116442 +0000 UTC m=+0.016186610 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:56:49.671 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 podman[31149]: 2024-09-19 03:56:49.561697536 +0000 UTC m=+0.279767698 container init ffebeb9c157efaf2b4c8966bb0752a35dbe8e9dcd503975cdc838f92f57d806f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T03:56:49.671 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 podman[31149]: 2024-09-19 03:56:49.564643276 +0000 UTC m=+0.282713425 container start ffebeb9c157efaf2b4c8966bb0752a35dbe8e9dcd503975cdc838f92f57d806f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T03:56:49.671 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 bash[31149]: ffebeb9c157efaf2b4c8966bb0752a35dbe8e9dcd503975cdc838f92f57d806f 2024-09-19T03:56:49.686 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T03:56:49.686 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-09-19T03:56:49.694 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T03:56:49.694 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-19T03:56:49.694 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-09-19T03:56:49.694 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T03:56:49.703 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T03:56:49.703 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-09-19T03:56:49.711 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T03:56:49.711 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-19T03:56:49.711 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-09-19T03:56:49.711 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T03:56:49.711 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-09-19T03:56:49.711 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-09-19T03:56:49.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 systemd[1]: Started Ceph mgr.x for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T03:56:49.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:49.691+0000 7f46f20b5040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T03:56:49.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:49.755+0000 7f46f20b5040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T03:56:49.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:49.912+0000 7f46f20b5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T03:56:50.208 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:49 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:49.984+0000 7f46f20b5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T03:56:50.208 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:50 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:50.158+0000 7f46f20b5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:56:50.502 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:50 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3962062710' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T03:56:50.502 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:50 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:50.309+0000 7f46f20b5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T03:56:50.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T03:56:50.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T03:56:50.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:56:50.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T03:56:50.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T03:56:50.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T03:56:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T03:56:42:596148+0000", 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T03:56:50.735 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T03:56:42.596824+0000", 2024-09-19T03:56:50.736 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.119+0000 7f2c8e216640 1 Processor -- start 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.119+0000 7f2c8e216640 1 -- start start 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.120+0000 7f2c8e216640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8810a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.120+0000 7f2c8e216640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8807bed0 con 0x7f2c881080f0 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.120+0000 7f2c877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8810a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.120+0000 7f2c877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8810a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46174/0 (socket says 172.21.15.19:46174) 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.120+0000 7f2c877fe640 1 -- 172.21.15.19:0/2411813351 learned_addr learned my addr 172.21.15.19:0/2411813351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.120+0000 7f2c877fe640 1 -- 172.21.15.19:0/2411813351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8810aa20 con 0x7f2c881080f0 2024-09-19T03:56:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.121+0000 7f2c877fe640 1 --2- 172.21.15.19:0/2411813351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8810a4e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009820 tx=0x7f2c7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa70acf06d13cb0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.121+0000 7f2c867fc640 1 -- 172.21.15.19:0/2411813351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70037070 con 0x7f2c881080f0 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.121+0000 7f2c867fc640 1 -- 172.21.15.19:0/2411813351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c70032900 con 0x7f2c881080f0 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.121+0000 7f2c867fc640 1 -- 172.21.15.19:0/2411813351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70030850 con 0x7f2c881080f0 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.122+0000 7f2c8e216640 1 -- 172.21.15.19:0/2411813351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 msgr2=0x7f2c8810a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.122+0000 7f2c8e216640 1 --2- 172.21.15.19:0/2411813351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8810a4e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009820 tx=0x7f2c7002a270 comp rx=0 tx=0).stop 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.122+0000 7f2c8e216640 1 -- 172.21.15.19:0/2411813351 shutdown_connections 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.122+0000 7f2c8e216640 1 --2- 172.21.15.19:0/2411813351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8810a4e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.122+0000 7f2c8e216640 1 -- 172.21.15.19:0/2411813351 >> 172.21.15.19:0/2411813351 conn(0x7f2c88101d10 msgr2=0x7f2c88104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.123+0000 7f2c8e216640 1 -- 172.21.15.19:0/2411813351 shutdown_connections 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.123+0000 7f2c8e216640 1 -- 172.21.15.19:0/2411813351 wait complete. 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.123+0000 7f2c8e216640 1 Processor -- start 2024-09-19T03:56:50.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.124+0000 7f2c8e216640 1 -- start start 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.124+0000 7f2c8e216640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.124+0000 7f2c8e216640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8810bac0 con 0x7f2c881080f0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.124+0000 7f2c877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.124+0000 7f2c877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46180/0 (socket says 172.21.15.19:46180) 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.124+0000 7f2c877fe640 1 -- 172.21.15.19:0/3962062710 learned_addr learned my addr 172.21.15.19:0/3962062710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.124+0000 7f2c877fe640 1 -- 172.21.15.19:0/3962062710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8819bbb0 con 0x7f2c881080f0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.125+0000 7f2c877fe640 1 --2- 172.21.15.19:0/3962062710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8819b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2c70032ce0 tx=0x7f2c70032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.125+0000 7f2c84ff9640 1 -- 172.21.15.19:0/3962062710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70042070 con 0x7f2c881080f0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.125+0000 7f2c84ff9640 1 -- 172.21.15.19:0/3962062710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c700308c0 con 0x7f2c881080f0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.125+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8819be40 con 0x7f2c881080f0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.125+0000 7f2c84ff9640 1 -- 172.21.15.19:0/3962062710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70037040 con 0x7f2c881080f0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.125+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8819eb30 con 0x7f2c881080f0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.125+0000 7f2c84ff9640 1 -- 172.21.15.19:0/3962062710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c70045070 con 0x7f2c881080f0 2024-09-19T03:56:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.125+0000 7f2c84ff9640 1 -- 172.21.15.19:0/3962062710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c700463b0 con 0x7f2c881080f0 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.126+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c88108550 con 0x7f2c881080f0 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.128+0000 7f2c84ff9640 1 -- 172.21.15.19:0/3962062710 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2c70047440 con 0x7f2c881080f0 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.206+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2c8819ed00 con 0x7f2c881080f0 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.207+0000 7f2c84ff9640 1 -- 172.21.15.19:0/3962062710 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f2c70047620 con 0x7f2c881080f0 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.208+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 msgr2=0x7f2c8819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.208+0000 7f2c8e216640 1 --2- 172.21.15.19:0/3962062710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8819b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2c70032ce0 tx=0x7f2c70032d10 comp rx=0 tx=0).stop 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.208+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 shutdown_connections 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.208+0000 7f2c8e216640 1 --2- 172.21.15.19:0/3962062710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c881080f0 0x7f2c8819b670 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.208+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 >> 172.21.15.19:0/3962062710 conn(0x7f2c88101d10 msgr2=0x7f2c881028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.208+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 shutdown_connections 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:50.208+0000 7f2c8e216640 1 -- 172.21.15.19:0/3962062710 wait complete. 2024-09-19T03:56:50.740 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-09-19T03:56:50.902 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:50 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:50.642+0000 7f46f20b5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T03:56:50.903 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:50 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:50.707+0000 7f46f20b5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T03:56:50.903 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:50 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:50.901+0000 7f46f20b5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T03:56:52.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:51 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:51.823+0000 7f46f20b5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T03:56:52.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:51 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:51.893+0000 7f46f20b5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T03:56:52.914 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:52 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:52.647+0000 7f46f20b5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T03:56:52.914 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:52 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:52.771+0000 7f46f20b5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T03:56:53.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:52 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:52.924+0000 7f46f20b5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:56:53.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:52 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:52.990+0000 7f46f20b5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T03:56:53.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.151+0000 7f46f20b5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T03:56:53.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2341657497' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T03:56:53.459 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.228+0000 7f46f20b5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T03:56:53.459 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.291+0000 7f46f20b5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T03:56:53.459 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.457+0000 7f46f20b5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T03:56:53.753 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.532+0000 7f46f20b5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T03:56:53.753 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.602+0000 7f46f20b5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T03:56:53.753 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.689+0000 7f46f20b5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T03:56:53.753 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.751+0000 7f46f20b5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T03:56:53.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:56:53.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-19T03:56:53.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T03:56:53.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T03:56:53.807 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T03:56:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T03:56:53.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T03:56:42:596148+0000", 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T03:56:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:56:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T03:56:42.596824+0000", 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.116+0000 7f8ed4177640 1 Processor -- start 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.116+0000 7f8ed4177640 1 -- start start 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.116+0000 7f8ed4177640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc106380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.116+0000 7f8ed4177640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc07be50 con 0x7f8ecc105f80 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.116+0000 7f8ed1f13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:53.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.117+0000 7f8ed1f13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc106380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51688/0 (socket says 172.21.15.19:51688) 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.117+0000 7f8ed1f13640 1 -- 172.21.15.19:0/3592054245 learned_addr learned my addr 172.21.15.19:0/3592054245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.117+0000 7f8ed1f13640 1 -- 172.21.15.19:0/3592054245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1068c0 con 0x7f8ecc105f80 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.117+0000 7f8ed1f13640 1 --2- 172.21.15.19:0/3592054245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc106380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009820 tx=0x7f8ec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc01ffb8fe72f83a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.118+0000 7f8ed0f11640 1 -- 172.21.15.19:0/3592054245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec8037070 con 0x7f8ecc105f80 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.118+0000 7f8ed0f11640 1 -- 172.21.15.19:0/3592054245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8ec8032900 con 0x7f8ecc105f80 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.118+0000 7f8ed0f11640 1 -- 172.21.15.19:0/3592054245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec8030850 con 0x7f8ecc105f80 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.118+0000 7f8ed4177640 1 -- 172.21.15.19:0/3592054245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 msgr2=0x7f8ecc106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.118+0000 7f8ed4177640 1 --2- 172.21.15.19:0/3592054245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc106380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009820 tx=0x7f8ec802a270 comp rx=0 tx=0).stop 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.118+0000 7f8ed4177640 1 -- 172.21.15.19:0/3592054245 shutdown_connections 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.118+0000 7f8ed4177640 1 --2- 172.21.15.19:0/3592054245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc106380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.118+0000 7f8ed4177640 1 -- 172.21.15.19:0/3592054245 >> 172.21.15.19:0/3592054245 conn(0x7f8ecc101d50 msgr2=0x7f8ecc104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:53.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.119+0000 7f8ed4177640 1 -- 172.21.15.19:0/3592054245 shutdown_connections 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.119+0000 7f8ed4177640 1 -- 172.21.15.19:0/3592054245 wait complete. 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.119+0000 7f8ed4177640 1 Processor -- start 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.119+0000 7f8ed4177640 1 -- start start 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.119+0000 7f8ed4177640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.119+0000 7f8ed4177640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1079c0 con 0x7f8ecc105f80 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.119+0000 7f8ed1f13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ed1f13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc19d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51690/0 (socket says 172.21.15.19:51690) 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ed1f13640 1 -- 172.21.15.19:0/2341657497 learned_addr learned my addr 172.21.15.19:0/2341657497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ed1f13640 1 -- 172.21.15.19:0/2341657497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc19de20 con 0x7f8ecc105f80 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ed1f13640 1 --2- 172.21.15.19:0/2341657497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc19d8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8031000 tx=0x7f8ec80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ec2ffd640 1 -- 172.21.15.19:0/2341657497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec8042070 con 0x7f8ecc105f80 2024-09-19T03:56:53.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc19e0b0 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ec2ffd640 1 -- 172.21.15.19:0/2341657497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8ec8032be0 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ec2ffd640 1 -- 172.21.15.19:0/2341657497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec8037040 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.120+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1a0da0 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.121+0000 7f8ec2ffd640 1 -- 172.21.15.19:0/2341657497 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ec8045070 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.121+0000 7f8ec2ffd640 1 -- 172.21.15.19:0/2341657497 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8ec8047a80 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.121+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ecc106380 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.123+0000 7f8ec2ffd640 1 -- 172.21.15.19:0/2341657497 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8ec8032d90 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.186+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8ecc1973b0 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.186+0000 7f8ec2ffd640 1 -- 172.21.15.19:0/2341657497 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f8ec803d430 con 0x7f8ecc105f80 2024-09-19T03:56:53.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.187+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 msgr2=0x7f8ecc19d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:56:53.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.187+0000 7f8ed4177640 1 --2- 172.21.15.19:0/2341657497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc19d8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8031000 tx=0x7f8ec80326e0 comp rx=0 tx=0).stop 2024-09-19T03:56:53.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.187+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 shutdown_connections 2024-09-19T03:56:53.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.187+0000 7f8ed4177640 1 --2- 172.21.15.19:0/2341657497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ecc105f80 0x7f8ecc19d8e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:56:53.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.187+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 >> 172.21.15.19:0/2341657497 conn(0x7f8ecc101d50 msgr2=0x7f8ecc194580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:56:53.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.187+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 shutdown_connections 2024-09-19T03:56:53.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:53.187+0000 7f8ed4177640 1 -- 172.21.15.19:0/2341657497 wait complete. 2024-09-19T03:56:53.823 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-09-19T03:56:54.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:53 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:53.814+0000 7f46f20b5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T03:56:54.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:56:54.644+0000 7f46f20b5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T03:56:54.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: Activating manager daemon x 2024-09-19T03:56:54.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: mgrmap e2: x(active, starting, since 0.00324253s) 2024-09-19T03:56:54.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T03:56:54.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T03:56:54.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T03:56:54.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T03:56:54.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T03:56:54.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: Manager daemon x is now available 2024-09-19T03:56:54.933 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T03:56:54.933 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T03:56:54.933 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' 2024-09-19T03:56:54.933 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' 2024-09-19T03:56:54.933 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:54 smithi019 ceph-mon[30938]: from='mgr.14100 172.21.15.19:0/45942461' entity='mgr.x' 2024-09-19T03:56:56.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:56 smithi019 ceph-mon[30938]: mgrmap e3: x(active, since 1.00761s) 2024-09-19T03:56:56.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:56 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4050755265' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T03:56:57.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:56:57 smithi019 ceph-mon[30938]: mgrmap e4: x(active, since 2s) 2024-09-19T03:57:02.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:57:02.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-19T03:57:02.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T03:57:02.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T03:57:02.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T03:57:02.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T03:57:02.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T03:57:02.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:57:02.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T03:57:02.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T03:57:02.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-09-19T03:57:02.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:57:02.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T03:57:02.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T03:57:02.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:57:02.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-19T03:57:02.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T03:57:02.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:57:02.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T03:57:02.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T03:57:02.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:57:02.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T03:57:02.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:57:02.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T03:57:02.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T03:57:02.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T03:57:02.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T03:57:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T03:57:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T03:57:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:57:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T03:57:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T03:56:42:596148+0000", 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T03:57:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T03:57:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T03:56:42.596824+0000", 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.211+0000 7fc54e573640 1 Processor -- start 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.211+0000 7fc54e573640 1 -- start start 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.211+0000 7fc54e573640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc548106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.212+0000 7fc54e573640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc548106f50 con 0x7fc548106580 2024-09-19T03:57:02.530 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.212+0000 7fc547fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc548106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.212+0000 7fc547fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc548106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51746/0 (socket says 172.21.15.19:51746) 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.212+0000 7fc547fff640 1 -- 172.21.15.19:0/3453240068 learned_addr learned my addr 172.21.15.19:0/3453240068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.212+0000 7fc547fff640 1 -- 172.21.15.19:0/3453240068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc548107780 con 0x7fc548106580 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.213+0000 7fc547fff640 1 --2- 172.21.15.19:0/3453240068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc548106980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc538009820 tx=0x7fc53802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5008a73f1016ede7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.213+0000 7fc546ffd640 1 -- 172.21.15.19:0/3453240068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc538037070 con 0x7fc548106580 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.213+0000 7fc546ffd640 1 -- 172.21.15.19:0/3453240068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc538032900 con 0x7fc548106580 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.213+0000 7fc546ffd640 1 -- 172.21.15.19:0/3453240068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc538030850 con 0x7fc548106580 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.213+0000 7fc54e573640 1 -- 172.21.15.19:0/3453240068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 msgr2=0x7fc548106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.213+0000 7fc54e573640 1 --2- 172.21.15.19:0/3453240068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc548106980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc538009820 tx=0x7fc53802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.214+0000 7fc54e573640 1 -- 172.21.15.19:0/3453240068 shutdown_connections 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.214+0000 7fc54e573640 1 --2- 172.21.15.19:0/3453240068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc548106980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.214+0000 7fc54e573640 1 -- 172.21.15.19:0/3453240068 >> 172.21.15.19:0/3453240068 conn(0x7fc548101d10 msgr2=0x7fc548104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:02.531 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.214+0000 7fc54e573640 1 -- 172.21.15.19:0/3453240068 shutdown_connections 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.214+0000 7fc54e573640 1 -- 172.21.15.19:0/3453240068 wait complete. 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.214+0000 7fc54e573640 1 Processor -- start 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.214+0000 7fc54e573640 1 -- start start 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc54e573640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc5481917b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc54e573640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc54810ab30 con 0x7fc548106580 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc547fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc5481917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc547fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc5481917b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51750/0 (socket says 172.21.15.19:51750) 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc547fff640 1 -- 172.21.15.19:0/4050755265 learned_addr learned my addr 172.21.15.19:0/4050755265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc547fff640 1 -- 172.21.15.19:0/4050755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc548191cf0 con 0x7fc548106580 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc547fff640 1 --2- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc5481917b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc5380097f0 tx=0x7fc538030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc5457fa640 1 -- 172.21.15.19:0/4050755265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc538037040 con 0x7fc548106580 2024-09-19T03:57:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54818fee0 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc5457fa640 1 -- 172.21.15.19:0/4050755265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc538040070 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.215+0000 7fc5457fa640 1 -- 172.21.15.19:0/4050755265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc53803a390 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.216+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481903c0 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.216+0000 7fc5457fa640 1 -- 172.21.15.19:0/4050755265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7fc53803a630 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.216+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54810acb0 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.217+0000 7fc5457fa640 1 --2- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7fc530039050 0x7fc53003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.217+0000 7fc5457fa640 1 -- 172.21.15.19:0/4050755265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc53806c310 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.217+0000 7fc5477fe640 1 --2- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7fc530039050 0x7fc53003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.217+0000 7fc5477fe640 1 --2- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7fc530039050 0x7fc53003b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0098c0 tx=0x7fc53c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.222+0000 7fc5457fa640 1 -- 172.21.15.19:0/4050755265 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc53803a9d0 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.463+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc548195530 con 0x7fc548106580 2024-09-19T03:57:02.533 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.464+0000 7fc5457fa640 1 -- 172.21.15.19:0/4050755265 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fc538042d40 con 0x7fc548106580 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7fc530039050 msgr2=0x7fc53003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 --2- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7fc530039050 0x7fc53003b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0098c0 tx=0x7fc53c006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 msgr2=0x7fc5481917b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 --2- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc5481917b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc5380097f0 tx=0x7fc538030d80 comp rx=0 tx=0).stop 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 shutdown_connections 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 --2- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7fc530039050 0x7fc53003b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 --2- 172.21.15.19:0/4050755265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548106580 0x7fc5481917b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 >> 172.21.15.19:0/4050755265 conn(0x7fc548101d10 msgr2=0x7fc548102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 shutdown_connections 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:56:56.466+0000 7fc54e573640 1 -- 172.21.15.19:0/4050755265 wait complete. 2024-09-19T03:57:02.534 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-09-19T03:57:05.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:05 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3217834310' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T03:57:09.211 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-19T03:57:09.219 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.466+0000 7f1ab3a86640 1 Processor -- start 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.467+0000 7f1ab3a86640 1 -- start start 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.467+0000 7f1ab3a86640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.467+0000 7f1ab3a86640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac1080f0 con 0x7f1aac107720 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.467+0000 7f1ab1822640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.468+0000 7f1ab1822640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53746/0 (socket says 172.21.15.19:53746) 2024-09-19T03:57:09.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.468+0000 7f1ab1822640 1 -- 172.21.15.19:0/4088081913 learned_addr learned my addr 172.21.15.19:0/4088081913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.468+0000 7f1ab1822640 1 -- 172.21.15.19:0/4088081913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac108920 con 0x7f1aac107720 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab1822640 1 --2- 172.21.15.19:0/4088081913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac107b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009820 tx=0x7f1aa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4de5fd913578d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab0820640 1 -- 172.21.15.19:0/4088081913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8037070 con 0x7f1aac107720 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab0820640 1 -- 172.21.15.19:0/4088081913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1aa8032900 con 0x7f1aac107720 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab0820640 1 -- 172.21.15.19:0/4088081913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8030850 con 0x7f1aac107720 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab3a86640 1 -- 172.21.15.19:0/4088081913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 msgr2=0x7f1aac107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab3a86640 1 --2- 172.21.15.19:0/4088081913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac107b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009820 tx=0x7f1aa802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab3a86640 1 -- 172.21.15.19:0/4088081913 shutdown_connections 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab3a86640 1 --2- 172.21.15.19:0/4088081913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac107b20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.469+0000 7f1ab3a86640 1 -- 172.21.15.19:0/4088081913 >> 172.21.15.19:0/4088081913 conn(0x7f1aac102ed0 msgr2=0x7f1aac1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.470+0000 7f1ab3a86640 1 -- 172.21.15.19:0/4088081913 shutdown_connections 2024-09-19T03:57:09.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.470+0000 7f1ab3a86640 1 -- 172.21.15.19:0/4088081913 wait complete. 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.470+0000 7f1ab3a86640 1 Processor -- start 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.470+0000 7f1ab3a86640 1 -- start start 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab3a86640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab3a86640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac10bcd0 con 0x7f1aac107720 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab1822640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab1822640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac19b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53754/0 (socket says 172.21.15.19:53754) 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab1822640 1 -- 172.21.15.19:0/3217834310 learned_addr learned my addr 172.21.15.19:0/3217834310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab1822640 1 -- 172.21.15.19:0/3217834310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac19bc40 con 0x7f1aac107720 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab1822640 1 --2- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac19b700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009940 tx=0x7f1aa80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1a9effd640 1 -- 172.21.15.19:0/3217834310 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8042070 con 0x7f1aac107720 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1a9effd640 1 -- 172.21.15.19:0/3217834310 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1aa8032be0 con 0x7f1aac107720 2024-09-19T03:57:09.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aac19bed0 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1a9effd640 1 -- 172.21.15.19:0/3217834310 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8037040 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.471+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aac19ebc0 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.472+0000 7f1a9effd640 1 -- 172.21.15.19:0/3217834310 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f1aa8045070 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.473+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aac107ba0 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.473+0000 7f1a9effd640 1 --2- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f1a980391b0 0x7f1a9803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.473+0000 7f1a9effd640 1 -- 172.21.15.19:0/3217834310 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1aa806c080 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.473+0000 7f1ab1021640 1 --2- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f1a980391b0 0x7f1a9803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.473+0000 7f1ab1021640 1 --2- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f1a980391b0 0x7f1a9803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00098c0 tx=0x7f1aa0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.478+0000 7f1a9effd640 1 -- 172.21.15.19:0/3217834310 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1aa804b170 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.657+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1aac1a03c0 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.658+0000 7f1a9effd640 1 -- 172.21.15.19:0/3217834310 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f1aa8032d90 con 0x7f1aac107720 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.660+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f1a980391b0 msgr2=0x7f1a9803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:09.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.660+0000 7f1ab3a86640 1 --2- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f1a980391b0 0x7f1a9803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00098c0 tx=0x7f1aa0006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.660+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 msgr2=0x7f1aac19b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.660+0000 7f1ab3a86640 1 --2- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac19b700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009940 tx=0x7f1aa80326e0 comp rx=0 tx=0).stop 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.661+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 shutdown_connections 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.661+0000 7f1ab3a86640 1 --2- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f1a980391b0 0x7f1a9803b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.661+0000 7f1ab3a86640 1 --2- 172.21.15.19:0/3217834310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aac107720 0x7f1aac19b700 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.661+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 >> 172.21.15.19:0/3217834310 conn(0x7f1aac102ed0 msgr2=0x7f1aac103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.661+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 shutdown_connections 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:05.661+0000 7f1ab3a86640 1 -- 172.21.15.19:0/3217834310 wait complete. 2024-09-19T03:57:09.224 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-09-19T03:57:10.266 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:09 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1029094679' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T03:57:10.266 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: ignoring --setuser ceph since I am not root 2024-09-19T03:57:10.266 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: ignoring --setgroup ceph since I am not root 2024-09-19T03:57:10.267 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:10.089+0000 7f12a34ab040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T03:57:10.267 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:10.149+0000 7f12a34ab040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T03:57:10.533 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:10.308+0000 7f12a34ab040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T03:57:10.534 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:10.378+0000 7f12a34ab040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T03:57:10.534 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:10.532+0000 7f12a34ab040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:57:10.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.660+0000 7f57e5957640 1 Processor -- start 2024-09-19T03:57:10.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.660+0000 7f57e5957640 1 -- start start 2024-09-19T03:57:10.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.660+0000 7f57e5957640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:10.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.660+0000 7f57e5957640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57e007c910 con 0x7f57e007dee0 2024-09-19T03:57:10.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.660+0000 7f57deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:10.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.660+0000 7f57deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53756/0 (socket says 172.21.15.19:53756) 2024-09-19T03:57:10.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.660+0000 7f57deffd640 1 -- 172.21.15.19:0/3987080808 learned_addr learned my addr 172.21.15.19:0/3987080808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.660+0000 7f57deffd640 1 -- 172.21.15.19:0/3987080808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e007ca90 con 0x7f57e007dee0 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.661+0000 7f57deffd640 1 --2- 172.21.15.19:0/3987080808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e007c340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009820 tx=0x7f57d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f632ee993ad5538c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.661+0000 7f57de7fc640 1 -- 172.21.15.19:0/3987080808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d0037070 con 0x7f57e007dee0 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.661+0000 7f57de7fc640 1 -- 172.21.15.19:0/3987080808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57d0032900 con 0x7f57e007dee0 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.661+0000 7f57de7fc640 1 -- 172.21.15.19:0/3987080808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d0030850 con 0x7f57e007dee0 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.662+0000 7f57e5957640 1 -- 172.21.15.19:0/3987080808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 msgr2=0x7f57e007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.662+0000 7f57e5957640 1 --2- 172.21.15.19:0/3987080808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e007c340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009820 tx=0x7f57d002a270 comp rx=0 tx=0).stop 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.662+0000 7f57e5957640 1 -- 172.21.15.19:0/3987080808 shutdown_connections 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.662+0000 7f57e5957640 1 --2- 172.21.15.19:0/3987080808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e007c340 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.662+0000 7f57e5957640 1 -- 172.21.15.19:0/3987080808 >> 172.21.15.19:0/3987080808 conn(0x7f57e0102ed0 msgr2=0x7f57e01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.662+0000 7f57e5957640 1 -- 172.21.15.19:0/3987080808 shutdown_connections 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.662+0000 7f57e5957640 1 -- 172.21.15.19:0/3987080808 wait complete. 2024-09-19T03:57:10.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.663+0000 7f57e5957640 1 Processor -- start 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.663+0000 7f57e5957640 1 -- start start 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.663+0000 7f57e5957640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.663+0000 7f57e5957640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010dd10 con 0x7f57e007dee0 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.663+0000 7f57deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.663+0000 7f57deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53764/0 (socket says 172.21.15.19:53764) 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.663+0000 7f57deffd640 1 -- 172.21.15.19:0/1029094679 learned_addr learned my addr 172.21.15.19:0/1029094679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.664+0000 7f57deffd640 1 -- 172.21.15.19:0/1029094679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e019de40 con 0x7f57e007dee0 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.664+0000 7f57deffd640 1 --2- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e019d900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f57d00097f0 tx=0x7f57d0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.664+0000 7f57e4955640 1 -- 172.21.15.19:0/1029094679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d0042070 con 0x7f57e007dee0 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.664+0000 7f57e4955640 1 -- 172.21.15.19:0/1029094679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57d0032ca0 con 0x7f57e007dee0 2024-09-19T03:57:10.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.664+0000 7f57e4955640 1 -- 172.21.15.19:0/1029094679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d0037040 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.664+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e019e0d0 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.664+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e01a0dc0 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.665+0000 7f57e4955640 1 -- 172.21.15.19:0/1029094679 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f57d0045070 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.665+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e007c3c0 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.665+0000 7f57e4955640 1 --2- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f57cc0391b0 0x7f57cc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.665+0000 7f57e4955640 1 -- 172.21.15.19:0/1029094679 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57d006c0a0 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.665+0000 7f57d67fc640 1 --2- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f57cc0391b0 0x7f57cc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.666+0000 7f57d67fc640 1 --2- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f57cc0391b0 0x7f57cc03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57c80098c0 tx=0x7f57c8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.670+0000 7f57e4955640 1 -- 172.21.15.19:0/1029094679 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f57d004b170 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.895+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f57e01a1440 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.946+0000 7f57e4955640 1 -- 172.21.15.19:0/1029094679 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f57d003d430 con 0x7f57e007dee0 2024-09-19T03:57:10.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.948+0000 7f57e4955640 1 -- 172.21.15.19:0/1029094679 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f57d006e0c0 con 0x7f57e007dee0 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f57cc0391b0 msgr2=0x7f57cc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 --2- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f57cc0391b0 0x7f57cc03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57c80098c0 tx=0x7f57c8006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 msgr2=0x7f57e019d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 --2- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e019d900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f57d00097f0 tx=0x7f57d0030ec0 comp rx=0 tx=0).stop 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 shutdown_connections 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 --2- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f57cc0391b0 0x7f57cc03b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 --2- 172.21.15.19:0/1029094679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e007dee0 0x7f57e019d900 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 >> 172.21.15.19:0/1029094679 conn(0x7f57e0102ed0 msgr2=0x7f57e0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 shutdown_connections 2024-09-19T03:57:10.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:09.949+0000 7f57e5957640 1 -- 172.21.15.19:0/1029094679 wait complete. 2024-09-19T03:57:10.853 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:10.688+0000 7f12a34ab040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T03:57:11.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:10 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1029094679' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T03:57:11.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:10 smithi019 ceph-mon[30938]: mgrmap e5: x(active, since 15s) 2024-09-19T03:57:11.140 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:11 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:11.025+0000 7f12a34ab040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T03:57:11.140 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:11 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:11.088+0000 7f12a34ab040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T03:57:11.431 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:11 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:11.286+0000 7f12a34ab040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T03:57:11.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.932+0000 7f04ed910640 1 Processor -- start 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.932+0000 7f04ed910640 1 -- start start 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.932+0000 7f04ed910640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e810a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.932+0000 7f04ed910640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04e807be50 con 0x7f04e81081c0 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.932+0000 7f04e6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e810a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.932+0000 7f04e6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e810a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53790/0 (socket says 172.21.15.19:53790) 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.932+0000 7f04e6ffd640 1 -- 172.21.15.19:0/1659023513 learned_addr learned my addr 172.21.15.19:0/1659023513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.933+0000 7f04e6ffd640 1 -- 172.21.15.19:0/1659023513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e810aaf0 con 0x7f04e81081c0 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.933+0000 7f04e6ffd640 1 --2- 172.21.15.19:0/1659023513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e810a5b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009820 tx=0x7f04d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203a476e23669427 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.933+0000 7f04e5ffb640 1 -- 172.21.15.19:0/1659023513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8037070 con 0x7f04e81081c0 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.933+0000 7f04e5ffb640 1 -- 172.21.15.19:0/1659023513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f04d8032900 con 0x7f04e81081c0 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.933+0000 7f04e5ffb640 1 -- 172.21.15.19:0/1659023513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8030850 con 0x7f04e81081c0 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.934+0000 7f04ed910640 1 -- 172.21.15.19:0/1659023513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 msgr2=0x7f04e810a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.934+0000 7f04ed910640 1 --2- 172.21.15.19:0/1659023513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e810a5b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009820 tx=0x7f04d802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.934+0000 7f04ed910640 1 -- 172.21.15.19:0/1659023513 shutdown_connections 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.934+0000 7f04ed910640 1 --2- 172.21.15.19:0/1659023513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e810a5b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.934+0000 7f04ed910640 1 -- 172.21.15.19:0/1659023513 >> 172.21.15.19:0/1659023513 conn(0x7f04e8101da0 msgr2=0x7f04e81041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.934+0000 7f04ed910640 1 -- 172.21.15.19:0/1659023513 shutdown_connections 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.934+0000 7f04ed910640 1 -- 172.21.15.19:0/1659023513 wait complete. 2024-09-19T03:57:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.935+0000 7f04ed910640 1 Processor -- start 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.935+0000 7f04ed910640 1 -- start start 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.935+0000 7f04ed910640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e8107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.935+0000 7f04ed910640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04e810bad0 con 0x7f04e81081c0 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.935+0000 7f04e6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e8107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.935+0000 7f04e6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e8107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53794/0 (socket says 172.21.15.19:53794) 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.935+0000 7f04e6ffd640 1 -- 172.21.15.19:0/1150769837 learned_addr learned my addr 172.21.15.19:0/1150769837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.936+0000 7f04e6ffd640 1 -- 172.21.15.19:0/1150769837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e8107f80 con 0x7f04e81081c0 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.936+0000 7f04e6ffd640 1 --2- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e8107a40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f04d8032ce0 tx=0x7f04d8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.936+0000 7f04ec90e640 1 -- 172.21.15.19:0/1150769837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8042070 con 0x7f04e81081c0 2024-09-19T03:57:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.936+0000 7f04ec90e640 1 -- 172.21.15.19:0/1150769837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f04d80308c0 con 0x7f04e81081c0 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.936+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e8106170 con 0x7f04e81081c0 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.936+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81066b0 con 0x7f04e81081c0 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.936+0000 7f04ec90e640 1 -- 172.21.15.19:0/1150769837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8037040 con 0x7f04e81081c0 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.937+0000 7f04ec90e640 1 -- 172.21.15.19:0/1150769837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f04d8045070 con 0x7f04e81081c0 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.937+0000 7f04ec90e640 1 --2- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f04d00391b0 0x7f04d003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.937+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e8108620 con 0x7f04e81081c0 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.937+0000 7f04e67fc640 1 -- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f04d00391b0 msgr2=0x7f04d003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3897870084 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.937+0000 7f04ec90e640 1 -- 172.21.15.19:0/1150769837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f04d806c3a0 con 0x7f04e81081c0 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.937+0000 7f04e67fc640 1 --2- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f04d00391b0 0x7f04d003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:10.942+0000 7f04ec90e640 1 -- 172.21.15.19:0/1150769837 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04d8039940 con 0x7f04e81081c0 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.138+0000 7f04e67fc640 1 -- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f04d00391b0 msgr2=0x7f04d003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3897870084 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.138+0000 7f04e67fc640 1 --2- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f04d00391b0 0x7f04d003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T03:57:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.164+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f04e8106930 con 0x7f04e81081c0 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.165+0000 7f04ec90e640 1 -- 172.21.15.19:0/1150769837 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f04d806be20 con 0x7f04e81081c0 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f04d00391b0 msgr2=0x7f04d003b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 --2- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f04d00391b0 0x7f04d003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 msgr2=0x7f04e8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 --2- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e8107a40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f04d8032ce0 tx=0x7f04d8032d10 comp rx=0 tx=0).stop 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 shutdown_connections 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 --2- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f04d00391b0 0x7f04d003b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 --2- 172.21.15.19:0/1150769837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e81081c0 0x7f04e8107a40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 >> 172.21.15.19:0/1150769837 conn(0x7f04e8101da0 msgr2=0x7f04e81028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 shutdown_connections 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:11.167+0000 7f04ed910640 1 -- 172.21.15.19:0/1150769837 wait complete. 2024-09-19T03:57:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-09-19T03:57:11.762 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-09-19T03:57:12.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:11 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1150769837' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T03:57:12.292 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:12 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:12.220+0000 7f12a34ab040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T03:57:12.293 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:12 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:12.291+0000 7f12a34ab040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T03:57:13.310 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.040+0000 7f12a34ab040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T03:57:13.310 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.160+0000 7f12a34ab040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T03:57:13.604 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.308+0000 7f12a34ab040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:57:13.605 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.372+0000 7f12a34ab040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T03:57:13.605 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.532+0000 7f12a34ab040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T03:57:13.605 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.603+0000 7f12a34ab040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T03:57:13.900 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.666+0000 7f12a34ab040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T03:57:13.900 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.831+0000 7f12a34ab040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T03:57:13.900 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.898+0000 7f12a34ab040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T03:57:14.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:13.971+0000 7f12a34ab040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T03:57:14.182 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:14.059+0000 7f12a34ab040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T03:57:14.182 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:14.121+0000 7f12a34ab040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T03:57:14.681 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:14.182+0000 7f12a34ab040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T03:57:15.431 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:15.010+0000 7f12a34ab040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T03:57:15.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: Active manager daemon x restarted 2024-09-19T03:57:15.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: Activating manager daemon x 2024-09-19T03:57:15.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T03:57:15.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: mgrmap e6: x(active, starting, since 0.00326298s) 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: Manager daemon x is now available 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:57:15.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T03:57:15.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:15 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T03:57:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:16 smithi019 ceph-mon[30938]: Found migration_current of "None". Setting to last migration. 2024-09-19T03:57:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:16 smithi019 ceph-mon[30938]: mgrmap e7: x(active, since 1.00529s) 2024-09-19T03:57:16.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-19T03:57:16.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T03:57:16.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T03:57:16.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-19T03:57:16.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.105+0000 7f0b0603b640 1 Processor -- start 2024-09-19T03:57:16.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.105+0000 7f0b0603b640 1 -- start start 2024-09-19T03:57:16.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.105+0000 7f0b0603b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0007bd10 0x7f0b0010a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:16.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.105+0000 7f0b0603b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0010a840 con 0x7f0b0007bd10 2024-09-19T03:57:16.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.105+0000 7f0aff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0007bd10 0x7f0b0010a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.105+0000 7f0aff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0007bd10 0x7f0b0010a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37548/0 (socket says 172.21.15.19:37548) 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.106+0000 7f0aff7fe640 1 -- 172.21.15.19:0/434468541 learned_addr learned my addr 172.21.15.19:0/434468541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.106+0000 7f0aff7fe640 1 -- 172.21.15.19:0/434468541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0010a9c0 con 0x7f0b0007bd10 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.106+0000 7f0aff7fe640 1 --2- 172.21.15.19:0/434468541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0007bd10 0x7f0b0010a270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009820 tx=0x7f0af002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad15465deb6086d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0afe7fc640 1 -- 172.21.15.19:0/434468541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af0037070 con 0x7f0b0007bd10 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0afe7fc640 1 -- 172.21.15.19:0/434468541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0af0032900 con 0x7f0b0007bd10 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0afe7fc640 1 -- 172.21.15.19:0/434468541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af0030850 con 0x7f0b0007bd10 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0b0603b640 1 -- 172.21.15.19:0/434468541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0007bd10 msgr2=0x7f0b0010a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0b0603b640 1 --2- 172.21.15.19:0/434468541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0007bd10 0x7f0b0010a270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009820 tx=0x7f0af002a270 comp rx=0 tx=0).stop 2024-09-19T03:57:16.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0b0603b640 1 -- 172.21.15.19:0/434468541 shutdown_connections 2024-09-19T03:57:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0b0603b640 1 --2- 172.21.15.19:0/434468541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0007bd10 0x7f0b0010a270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0b0603b640 1 -- 172.21.15.19:0/434468541 >> 172.21.15.19:0/434468541 conn(0x7f0b00101da0 msgr2=0x7f0b001041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.107+0000 7f0b0603b640 1 -- 172.21.15.19:0/434468541 shutdown_connections 2024-09-19T03:57:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.108+0000 7f0b0603b640 1 -- 172.21.15.19:0/434468541 wait complete. 2024-09-19T03:57:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.108+0000 7f0b0603b640 1 Processor -- start 2024-09-19T03:57:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.108+0000 7f0b0603b640 1 -- start start 2024-09-19T03:57:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0b0603b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0019b690 0x7f0b0019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0b0603b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0010bac0 con 0x7f0b0019b690 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0aff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0019b690 0x7f0b0019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0aff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0019b690 0x7f0b0019bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37564/0 (socket says 172.21.15.19:37564) 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0aff7fe640 1 -- 172.21.15.19:0/1151396217 learned_addr learned my addr 172.21.15.19:0/1151396217 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0aff7fe640 1 -- 172.21.15.19:0/1151396217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0019bff0 con 0x7f0b0019b690 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0aff7fe640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0019b690 0x7f0b0019bab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0af0031000 tx=0x7f0af00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af0042070 con 0x7f0b0019b690 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0af0032be0 con 0x7f0b0019b690 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0019c280 con 0x7f0b0019b690 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af0037040 con 0x7f0b0019b690 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.109+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0019ed60 con 0x7f0b0019b690 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.111+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f0af0045070 con 0x7f0b0019b690 2024-09-19T03:57:16.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.111+0000 7f0afcff9640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 0x7f0ae803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.111+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 --> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0ae803bd80 con 0x7f0ae80391b0 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.111+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0af006c140 con 0x7f0b0019b690 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.111+0000 7f0afeffd640 1 -- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 msgr2=0x7f0ae803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3897870084 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.111+0000 7f0afeffd640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 0x7f0ae803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.312+0000 7f0afeffd640 1 -- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 msgr2=0x7f0ae803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3897870084 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.312+0000 7f0afeffd640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 0x7f0ae803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.713+0000 7f0afeffd640 1 -- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 msgr2=0x7f0ae803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3897870084 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:12.713+0000 7f0afeffd640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 0x7f0ae803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:13.514+0000 7f0afeffd640 1 -- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 msgr2=0x7f0ae803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3897870084 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:13.514+0000 7f0afeffd640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 0x7f0ae803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:15.013+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f0af006acc0 con 0x7f0b0019b690 2024-09-19T03:57:16.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:15.013+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 msgr2=0x7f0ae803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:15.013+0000 7f0afcff9640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 0x7f0ae803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.014+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f0af003ccd0 con 0x7f0b0019b690 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.014+0000 7f0afcff9640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0ae803cae0 0x7f0ae803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.015+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0af0001d30 con 0x7f0ae803cae0 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.015+0000 7f0afeffd640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0ae803cae0 0x7f0ae803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.015+0000 7f0afeffd640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0ae803cae0 0x7f0ae803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0af4003980 tx=0x7f0af4006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.016+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mgr.14118 v2:172.21.15.19:6800/2229567042 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f0af0001d30 con 0x7f0ae803cae0 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0b001082e0 con 0x7f0ae803cae0 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0afcff9640 1 -- 172.21.15.19:0/1151396217 <== mgr.14118 v2:172.21.15.19:6800/2229567042 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f0b001082e0 con 0x7f0ae803cae0 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0ae803cae0 msgr2=0x7f0ae803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0ae803cae0 0x7f0ae803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0af4003980 tx=0x7f0af4006e10 comp rx=0 tx=0).stop 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0019b690 msgr2=0x7f0b0019bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0019b690 0x7f0b0019bab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0af0031000 tx=0x7f0af00326e0 comp rx=0 tx=0).stop 2024-09-19T03:57:16.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 shutdown_connections 2024-09-19T03:57:16.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0ae803cae0 0x7f0ae803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:16.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:6800/3897870084,v1:172.21.15.19:6801/3897870084] conn(0x7f0ae80391b0 0x7f0ae803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:16.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 --2- 172.21.15.19:0/1151396217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0019b690 0x7f0b0019bab0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:16.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 >> 172.21.15.19:0/1151396217 conn(0x7f0b00101da0 msgr2=0x7f0b001029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:16.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 shutdown_connections 2024-09-19T03:57:16.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:16.029+0000 7f0b0603b640 1 -- 172.21.15.19:0/1151396217 wait complete. 2024-09-19T03:57:16.723 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-09-19T03:57:16.723 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-09-19T03:57:17.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-mon[30938]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T03:57:17.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-mon[30938]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T03:57:17.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:17.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:17.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.087+0000 7fb34a321640 1 Processor -- start 2024-09-19T03:57:17.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.087+0000 7fb34a321640 1 -- start start 2024-09-19T03:57:17.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.087+0000 7fb34a321640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34410a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:17.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.087+0000 7fb34a321640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb34407bed0 con 0x7fb3441080f0 2024-09-19T03:57:17.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.087+0000 7fb343fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34410a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.087+0000 7fb343fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34410a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37620/0 (socket says 172.21.15.19:37620) 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.087+0000 7fb343fff640 1 -- 172.21.15.19:0/1283083573 learned_addr learned my addr 172.21.15.19:0/1283083573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.087+0000 7fb343fff640 1 -- 172.21.15.19:0/1283083573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34410aa20 con 0x7fb3441080f0 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.088+0000 7fb343fff640 1 --2- 172.21.15.19:0/1283083573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34410a4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb334009820 tx=0x7fb33402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b674ac3a741164 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.088+0000 7fb342ffd640 1 -- 172.21.15.19:0/1283083573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb334037070 con 0x7fb3441080f0 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.088+0000 7fb342ffd640 1 -- 172.21.15.19:0/1283083573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb334032900 con 0x7fb3441080f0 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.088+0000 7fb342ffd640 1 -- 172.21.15.19:0/1283083573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb334030850 con 0x7fb3441080f0 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.089+0000 7fb34a321640 1 -- 172.21.15.19:0/1283083573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 msgr2=0x7fb34410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.089+0000 7fb34a321640 1 --2- 172.21.15.19:0/1283083573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34410a4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb334009820 tx=0x7fb33402a270 comp rx=0 tx=0).stop 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.089+0000 7fb34a321640 1 -- 172.21.15.19:0/1283083573 shutdown_connections 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.089+0000 7fb34a321640 1 --2- 172.21.15.19:0/1283083573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34410a4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:17.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.089+0000 7fb34a321640 1 -- 172.21.15.19:0/1283083573 >> 172.21.15.19:0/1283083573 conn(0x7fb344101d10 msgr2=0x7fb344104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.089+0000 7fb34a321640 1 -- 172.21.15.19:0/1283083573 shutdown_connections 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.089+0000 7fb34a321640 1 -- 172.21.15.19:0/1283083573 wait complete. 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.090+0000 7fb34a321640 1 Processor -- start 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.090+0000 7fb34a321640 1 -- start start 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.090+0000 7fb34a321640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.090+0000 7fb34a321640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb34410bac0 con 0x7fb3441080f0 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.090+0000 7fb343fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.090+0000 7fb343fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34419b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37622/0 (socket says 172.21.15.19:37622) 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.090+0000 7fb343fff640 1 -- 172.21.15.19:0/2198521210 learned_addr learned my addr 172.21.15.19:0/2198521210 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.091+0000 7fb343fff640 1 -- 172.21.15.19:0/2198521210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34419bbb0 con 0x7fb3441080f0 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.091+0000 7fb343fff640 1 --2- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34419b670 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb334032ce0 tx=0x7fb334032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:17.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.091+0000 7fb3417fa640 1 -- 172.21.15.19:0/2198521210 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb334042070 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.091+0000 7fb3417fa640 1 -- 172.21.15.19:0/2198521210 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb3340308c0 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.091+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34419be40 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.091+0000 7fb3417fa640 1 -- 172.21.15.19:0/2198521210 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb334037040 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.091+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34419eb30 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.092+0000 7fb3417fa640 1 -- 172.21.15.19:0/2198521210 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb334045070 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.092+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb344108550 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.092+0000 7fb3417fa640 1 --2- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fb32c0391b0 0x7fb32c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.092+0000 7fb3417fa640 1 -- 172.21.15.19:0/2198521210 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb33406c340 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.092+0000 7fb3437fe640 1 --2- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fb32c0391b0 0x7fb32c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.093+0000 7fb3437fe640 1 --2- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fb32c0391b0 0x7fb32c03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3380098c0 tx=0x7fb338006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.097+0000 7fb3417fa640 1 -- 172.21.15.19:0/2198521210 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb33404b170 con 0x7fb3441080f0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.290+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb344103000 con 0x7fb32c0391b0 2024-09-19T03:57:17.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.295+0000 7fb3417fa640 1 -- 172.21.15.19:0/2198521210 <== mgr.14118 v2:172.21.15.19:6800/2229567042 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb344103000 con 0x7fb32c0391b0 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.297+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fb32c0391b0 msgr2=0x7fb32c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.297+0000 7fb34a321640 1 --2- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fb32c0391b0 0x7fb32c03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3380098c0 tx=0x7fb338006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.297+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 msgr2=0x7fb34419b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.297+0000 7fb34a321640 1 --2- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34419b670 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb334032ce0 tx=0x7fb334032d10 comp rx=0 tx=0).stop 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.297+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 shutdown_connections 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.297+0000 7fb34a321640 1 --2- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fb32c0391b0 0x7fb32c03b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.297+0000 7fb34a321640 1 --2- 172.21.15.19:0/2198521210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3441080f0 0x7fb34419b670 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.297+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 >> 172.21.15.19:0/2198521210 conn(0x7fb344101d10 msgr2=0x7fb3441028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.298+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 shutdown_connections 2024-09-19T03:57:17.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:17.298+0000 7fb34a321640 1 -- 172.21.15.19:0/2198521210 wait complete. 2024-09-19T03:57:17.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:17.764+0000 7f120ffd0640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:03:57:17] ENGINE Error in HTTPServer.serve 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: Traceback (most recent call last): 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: self._connections.run(self.expiration_interval) 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: self._run(expiration_interval) 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T03:57:17.932 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: s = self.context.wrap_socket( 2024-09-19T03:57:17.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T03:57:17.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: return self.sslsocket_class._create( 2024-09-19T03:57:17.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T03:57:17.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: self.do_handshake() 2024-09-19T03:57:17.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T03:57:17.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: self._sslobj.do_handshake() 2024-09-19T03:57:17.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T03:57:17.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:18.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: mgrmap e8: x(active, since 2s) 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:17] ENGINE Bus STARTING 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:17] ENGINE Serving on https://172.21.15.19:7150 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:17] ENGINE Error in HTTPServer.serve 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout: Traceback (most recent call last): 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._connections.run(self.expiration_interval) 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._run(expiration_interval) 2024-09-19T03:57:18.264 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: s = self.context.wrap_socket( 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: return self.sslsocket_class._create( 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: self.do_handshake() 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._sslobj.do_handshake() 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:17] ENGINE Serving on http://172.21.15.19:8765 2024-09-19T03:57:18.265 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:17] ENGINE Bus STARTED 2024-09-19T03:57:18.266 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:18 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:57:19.032 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.267+0000 7f5cd2e34640 1 Processor -- start 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.267+0000 7f5cd2e34640 1 -- start start 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.267+0000 7f5cd2e34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc10a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.267+0000 7f5cd2e34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc07be50 con 0x7f5ccc1081f0 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.267+0000 7f5cd0bd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc10a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.267+0000 7f5cd0bd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc10a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37628/0 (socket says 172.21.15.19:37628) 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.267+0000 7f5cd0bd0640 1 -- 172.21.15.19:0/3233983406 learned_addr learned my addr 172.21.15.19:0/3233983406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.267+0000 7f5cd0bd0640 1 -- 172.21.15.19:0/3233983406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc10ab20 con 0x7f5ccc1081f0 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.268+0000 7f5cd0bd0640 1 --2- 172.21.15.19:0/3233983406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc10a5e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009820 tx=0x7f5cc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c220d7246b98f405 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.268+0000 7f5ccb7fe640 1 -- 172.21.15.19:0/3233983406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc0037070 con 0x7f5ccc1081f0 2024-09-19T03:57:19.033 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.268+0000 7f5ccb7fe640 1 -- 172.21.15.19:0/3233983406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5cc0032900 con 0x7f5ccc1081f0 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.268+0000 7f5ccb7fe640 1 -- 172.21.15.19:0/3233983406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc0030850 con 0x7f5ccc1081f0 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.269+0000 7f5cd2e34640 1 -- 172.21.15.19:0/3233983406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 msgr2=0x7f5ccc10a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.269+0000 7f5cd2e34640 1 --2- 172.21.15.19:0/3233983406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc10a5e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009820 tx=0x7f5cc002a270 comp rx=0 tx=0).stop 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.269+0000 7f5cd2e34640 1 -- 172.21.15.19:0/3233983406 shutdown_connections 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.269+0000 7f5cd2e34640 1 --2- 172.21.15.19:0/3233983406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc10a5e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.269+0000 7f5cd2e34640 1 -- 172.21.15.19:0/3233983406 >> 172.21.15.19:0/3233983406 conn(0x7f5ccc101dd0 msgr2=0x7f5ccc1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.269+0000 7f5cd2e34640 1 -- 172.21.15.19:0/3233983406 shutdown_connections 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.269+0000 7f5cd2e34640 1 -- 172.21.15.19:0/3233983406 wait complete. 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.269+0000 7f5cd2e34640 1 Processor -- start 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd2e34640 1 -- start start 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd2e34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd2e34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10bb00 con 0x7f5ccc1081f0 2024-09-19T03:57:19.034 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd0bd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd0bd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc19b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37632/0 (socket says 172.21.15.19:37632) 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd0bd0640 1 -- 172.21.15.19:0/1835699564 learned_addr learned my addr 172.21.15.19:0/1835699564 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd0bd0640 1 -- 172.21.15.19:0/1835699564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc19bc70 con 0x7f5ccc1081f0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd0bd0640 1 --2- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc19b730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0031000 tx=0x7f5cc00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cc9ffb640 1 -- 172.21.15.19:0/1835699564 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc0042070 con 0x7f5ccc1081f0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.270+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc19bf00 con 0x7f5ccc1081f0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.271+0000 7f5cc9ffb640 1 -- 172.21.15.19:0/1835699564 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5cc0032be0 con 0x7f5ccc1081f0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.271+0000 7f5cc9ffb640 1 -- 172.21.15.19:0/1835699564 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc0037040 con 0x7f5ccc1081f0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.271+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc19ebf0 con 0x7f5ccc1081f0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.272+0000 7f5cc9ffb640 1 -- 172.21.15.19:0/1835699564 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5cc0045070 con 0x7f5ccc1081f0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.272+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc108650 con 0x7f5ccc1081f0 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.272+0000 7f5cc9ffb640 1 --2- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5cbc0391b0 0x7f5cbc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:19.035 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.272+0000 7f5cc9ffb640 1 -- 172.21.15.19:0/1835699564 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5cc006c350 con 0x7f5ccc1081f0 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.272+0000 7f5ccbfff640 1 --2- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5cbc0391b0 0x7f5cbc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.272+0000 7f5ccbfff640 1 --2- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5cbc0391b0 0x7f5cbc03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80098c0 tx=0x7f5cb8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.277+0000 7f5cc9ffb640 1 -- 172.21.15.19:0/1835699564 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5cc002f2a0 con 0x7f5ccc1081f0 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.453+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f5ccc1030a0 con 0x7f5cbc0391b0 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.454+0000 7f5cc9ffb640 1 -- 172.21.15.19:0/1835699564 <== mgr.14118 v2:172.21.15.19:6800/2229567042 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f5ccc1030a0 con 0x7f5cbc0391b0 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5cbc0391b0 msgr2=0x7f5cbc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 --2- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5cbc0391b0 0x7f5cbc03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80098c0 tx=0x7f5cb8006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 msgr2=0x7f5ccc19b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 --2- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc19b730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0031000 tx=0x7f5cc00326e0 comp rx=0 tx=0).stop 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 shutdown_connections 2024-09-19T03:57:19.036 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 --2- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5cbc0391b0 0x7f5cbc03b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:19.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 --2- 172.21.15.19:0/1835699564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ccc1081f0 0x7f5ccc19b730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:19.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 >> 172.21.15.19:0/1835699564 conn(0x7f5ccc101dd0 msgr2=0x7f5ccc102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:19.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 shutdown_connections 2024-09-19T03:57:19.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:18.456+0000 7f5cd2e34640 1 -- 172.21.15.19:0/1835699564 wait complete. 2024-09-19T03:57:19.037 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-09-19T03:57:19.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:19 smithi019 ceph-mon[30938]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:20.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-mon[30938]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-mon[30938]: Generating ssh key... 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: Generating public/private rsa key pair. 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: Your identification has been saved in /tmp/tmp75acnkt_/key 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: Your public key has been saved in /tmp/tmp75acnkt_/key.pub 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: The key fingerprint is: 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: SHA256:bMhgo41IFsoHYilhoPSfPE6Rfq6yWJT4dsNyhrzYqvA ceph-0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: The key's randomart image is: 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: +---[RSA 3072]----+ 2024-09-19T03:57:20.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: |=B. | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: |X.+ . | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: |++ o+o | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: |o.o==+oo | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: |..oo.Bo.S | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: | + = +. | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: |. B B . | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: |..*.* o | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: |o+E+o. | 2024-09-19T03:57:20.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: +----[SHA256]-----+ 2024-09-19T03:57:20.838 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.433+0000 7f9dce975640 1 Processor -- start 2024-09-19T03:57:20.838 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.434+0000 7f9dce975640 1 -- start start 2024-09-19T03:57:20.838 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.434+0000 7f9dce975640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:20.838 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.434+0000 7f9dce975640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc807be10 con 0x7f9dc81081b0 2024-09-19T03:57:20.838 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.434+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.434+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc810a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37640/0 (socket says 172.21.15.19:37640) 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.434+0000 7f9dc7fff640 1 -- 172.21.15.19:0/1973024897 learned_addr learned my addr 172.21.15.19:0/1973024897 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.434+0000 7f9dc7fff640 1 -- 172.21.15.19:0/1973024897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc810aae0 con 0x7f9dc81081b0 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.435+0000 7f9dc7fff640 1 --2- 172.21.15.19:0/1973024897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc810a5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009820 tx=0x7f9db802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17fc53e90e1b902 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.435+0000 7f9dc77fe640 1 -- 172.21.15.19:0/1973024897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db8037070 con 0x7f9dc81081b0 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.435+0000 7f9dc77fe640 1 -- 172.21.15.19:0/1973024897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9db8032900 con 0x7f9dc81081b0 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.435+0000 7f9dc77fe640 1 -- 172.21.15.19:0/1973024897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db8030850 con 0x7f9dc81081b0 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.436+0000 7f9dce975640 1 -- 172.21.15.19:0/1973024897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 msgr2=0x7f9dc810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.436+0000 7f9dce975640 1 --2- 172.21.15.19:0/1973024897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc810a5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009820 tx=0x7f9db802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.436+0000 7f9dce975640 1 -- 172.21.15.19:0/1973024897 shutdown_connections 2024-09-19T03:57:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.436+0000 7f9dce975640 1 --2- 172.21.15.19:0/1973024897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc810a5a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.436+0000 7f9dce975640 1 -- 172.21.15.19:0/1973024897 >> 172.21.15.19:0/1973024897 conn(0x7f9dc8101d10 msgr2=0x7f9dc8104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.436+0000 7f9dce975640 1 -- 172.21.15.19:0/1973024897 shutdown_connections 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.436+0000 7f9dce975640 1 -- 172.21.15.19:0/1973024897 wait complete. 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dce975640 1 Processor -- start 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dce975640 1 -- start start 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dce975640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc8107a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dce975640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc810bac0 con 0x7f9dc81081b0 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc8107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc8107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37652/0 (socket says 172.21.15.19:37652) 2024-09-19T03:57:20.840 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dc7fff640 1 -- 172.21.15.19:0/842774269 learned_addr learned my addr 172.21.15.19:0/842774269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dc7fff640 1 -- 172.21.15.19:0/842774269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc8107f70 con 0x7f9dc81081b0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dc7fff640 1 --2- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc8107a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097f0 tx=0x7f9db8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.437+0000 7f9dc5ffb640 1 -- 172.21.15.19:0/842774269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db8037040 con 0x7f9dc81081b0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.438+0000 7f9dc5ffb640 1 -- 172.21.15.19:0/842774269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9db8040070 con 0x7f9dc81081b0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.438+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc8106160 con 0x7f9dc81081b0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.438+0000 7f9dc5ffb640 1 -- 172.21.15.19:0/842774269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db803a390 con 0x7f9dc81081b0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.438+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc8106640 con 0x7f9dc81081b0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.438+0000 7f9dc5ffb640 1 -- 172.21.15.19:0/842774269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9db8032900 con 0x7f9dc81081b0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.438+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc8108610 con 0x7f9dc81081b0 2024-09-19T03:57:20.841 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.439+0000 7f9dc5ffb640 1 --2- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f9db4039200 0x7f9db403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.439+0000 7f9dc5ffb640 1 -- 172.21.15.19:0/842774269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9db806c740 con 0x7f9dc81081b0 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.439+0000 7f9dbffff640 1 --2- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f9db4039200 0x7f9db403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.439+0000 7f9dbffff640 1 --2- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f9db4039200 0x7f9db403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9db00098c0 tx=0x7f9db0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.443+0000 7f9dc5ffb640 1 -- 172.21.15.19:0/842774269 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9db803a5e0 con 0x7f9dc81081b0 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:19.625+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f9dc8106ef0 con 0x7f9db4039200 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.284+0000 7f9dc5ffb640 1 -- 172.21.15.19:0/842774269 <== mgr.14118 v2:172.21.15.19:6800/2229567042 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9dc8106ef0 con 0x7f9db4039200 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f9db4039200 msgr2=0x7f9db403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 --2- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f9db4039200 0x7f9db403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9db00098c0 tx=0x7f9db0006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 msgr2=0x7f9dc8107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 --2- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc8107a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097f0 tx=0x7f9db8030d80 comp rx=0 tx=0).stop 2024-09-19T03:57:20.842 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 shutdown_connections 2024-09-19T03:57:20.843 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 --2- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f9db4039200 0x7f9db403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:20.843 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 --2- 172.21.15.19:0/842774269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dc81081b0 0x7f9dc8107a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:20.843 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 >> 172.21.15.19:0/842774269 conn(0x7f9dc8101d10 msgr2=0x7f9dc81028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:20.843 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 shutdown_connections 2024-09-19T03:57:20.843 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:20.286+0000 7f9dce975640 1 -- 172.21.15.19:0/842774269 wait complete. 2024-09-19T03:57:21.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:21 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:21.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:21 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu0BwHdO5ZVgfJ6EjY87o5xBngC73LF1kcfsekz1IrsZC4kGdwLn1WRVIPGzDR2znF3uBJDbj78Sncc7xVFKjIcnMqZYpwLXTE9DXA02s/il1El/enDUN6lwvtOQ1eYdO+wjaEbO5lVdZnjczeW5j7XWQ+v2TZo5YB6Tf8SS6hsuNHNASZJgpj/rP40+W2WtYdsuBaEhherzP1cdJrmU8mJhek5BAQo3h9PHQ1vm8PZq+Ca4EnnWMQJhpeafSfbPmAD1kqbuwhDPucyU820VyG7znzaTq3Bxlyf9PcwLvvZQGye9V3B/fxuy/cNZrHMDq7gRxIYkb8DPffmnpx3zkOI9Djqe4ynDWwLhuRj1UNBnzyMyhBi8Z/kY934l1rrzZUe4lP0NzSpXxunJrgvGHMb2lXL6sHr7vdHbSsvctdhwqtVCD+tae35FWhh7zmGft7MYyBvmPlBnNaocduDm4lRdjJrl/01b8Jmje5HeYdbUTBkwyiFLk91PHJktATKY0= ceph-0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:57:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.248+0000 7f7969ff2640 1 Processor -- start 2024-09-19T03:57:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.248+0000 7f7969ff2640 1 -- start start 2024-09-19T03:57:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.248+0000 7f7969ff2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.248+0000 7f7969ff2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f796407c910 con 0x7f796407dee0 2024-09-19T03:57:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.249+0000 7f79637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.249+0000 7f79637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58966/0 (socket says 172.21.15.19:58966) 2024-09-19T03:57:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.249+0000 7f79637fe640 1 -- 172.21.15.19:0/3452061583 learned_addr learned my addr 172.21.15.19:0/3452061583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.249+0000 7f79637fe640 1 -- 172.21.15.19:0/3452061583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796407ca90 con 0x7f796407dee0 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.250+0000 7f79637fe640 1 --2- 172.21.15.19:0/3452061583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796407c340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7954009e80 tx=0x7f795402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43b6f53032973a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.250+0000 7f79627fc640 1 -- 172.21.15.19:0/3452061583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7954036070 con 0x7f796407dee0 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.250+0000 7f79627fc640 1 -- 172.21.15.19:0/3452061583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f79540318c0 con 0x7f796407dee0 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.250+0000 7f79627fc640 1 -- 172.21.15.19:0/3452061583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7954007ab0 con 0x7f796407dee0 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.250+0000 7f7969ff2640 1 -- 172.21.15.19:0/3452061583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 msgr2=0x7f796407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.250+0000 7f7969ff2640 1 --2- 172.21.15.19:0/3452061583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796407c340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7954009e80 tx=0x7f795402a7a0 comp rx=0 tx=0).stop 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.250+0000 7f7969ff2640 1 -- 172.21.15.19:0/3452061583 shutdown_connections 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.251+0000 7f7969ff2640 1 --2- 172.21.15.19:0/3452061583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796407c340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.251+0000 7f7969ff2640 1 -- 172.21.15.19:0/3452061583 >> 172.21.15.19:0/3452061583 conn(0x7f7964102ed0 msgr2=0x7f79641052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.251+0000 7f7969ff2640 1 -- 172.21.15.19:0/3452061583 shutdown_connections 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.251+0000 7f7969ff2640 1 -- 172.21.15.19:0/3452061583 wait complete. 2024-09-19T03:57:22.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.251+0000 7f7969ff2640 1 Processor -- start 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.251+0000 7f7969ff2640 1 -- start start 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f7969ff2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f7969ff2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f796410dd10 con 0x7f796407dee0 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f79637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f79637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796419d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58970/0 (socket says 172.21.15.19:58970) 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f79637fe640 1 -- 172.21.15.19:0/3487335910 learned_addr learned my addr 172.21.15.19:0/3487335910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f79637fe640 1 -- 172.21.15.19:0/3487335910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796419de40 con 0x7f796407dee0 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f79637fe640 1 --2- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796419d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7954007ce0 tx=0x7f7954031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f7960ff9640 1 -- 172.21.15.19:0/3487335910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7954036050 con 0x7f796407dee0 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f7960ff9640 1 -- 172.21.15.19:0/3487335910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7954042070 con 0x7f796407dee0 2024-09-19T03:57:22.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796419e0d0 con 0x7f796407dee0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f7960ff9640 1 -- 172.21.15.19:0/3487335910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f795403d8a0 con 0x7f796407dee0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.252+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79641a0dc0 con 0x7f796407dee0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.253+0000 7f7960ff9640 1 -- 172.21.15.19:0/3487335910 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f795403dba0 con 0x7f796407dee0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.253+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796407c3c0 con 0x7f796407dee0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.253+0000 7f7960ff9640 1 --2- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f79500391b0 0x7f795003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.253+0000 7f7960ff9640 1 -- 172.21.15.19:0/3487335910 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7954007e00 con 0x7f796407dee0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.254+0000 7f7962ffd640 1 --2- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f79500391b0 0x7f795003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.254+0000 7f7962ffd640 1 --2- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f79500391b0 0x7f795003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f794c0098c0 tx=0x7f794c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.258+0000 7f7960ff9640 1 -- 172.21.15.19:0/3487335910 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7954044310 con 0x7f796407dee0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.435+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f79641041d0 con 0x7f79500391b0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.435+0000 7f7960ff9640 1 -- 172.21.15.19:0/3487335910 <== mgr.14118 v2:172.21.15.19:6800/2229567042 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f79641041d0 con 0x7f79500391b0 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f79500391b0 msgr2=0x7f795003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:22.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 --2- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f79500391b0 0x7f795003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f794c0098c0 tx=0x7f794c006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 msgr2=0x7f796419d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 --2- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796419d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7954007ce0 tx=0x7f7954031ca0 comp rx=0 tx=0).stop 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 shutdown_connections 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 --2- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f79500391b0 0x7f795003b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 --2- 172.21.15.19:0/3487335910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f796407dee0 0x7f796419d900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 >> 172.21.15.19:0/3487335910 conn(0x7f7964102ed0 msgr2=0x7f7964103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 shutdown_connections 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:21.438+0000 7f7969ff2640 1 -- 172.21.15.19:0/3487335910 wait complete. 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T03:57:22.015 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T03:57:22.017 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-09-19T03:57:22.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:22 smithi019 ceph-mon[30938]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:23.636 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:23 smithi019 ceph-mon[30938]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:24.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:24 smithi019 ceph-mon[30938]: Deploying cephadm binary to smithi019 2024-09-19T03:57:27.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-09-19T03:57:27.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.398+0000 7f4ab75b7640 1 Processor -- start 2024-09-19T03:57:27.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.398+0000 7f4ab75b7640 1 -- start start 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.398+0000 7f4ab75b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab010a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.398+0000 7f4ab75b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab007bed0 con 0x7f4ab01080f0 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.399+0000 7f4ab5353640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab010a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.399+0000 7f4ab5353640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab010a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58986/0 (socket says 172.21.15.19:58986) 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.399+0000 7f4ab5353640 1 -- 172.21.15.19:0/3792495311 learned_addr learned my addr 172.21.15.19:0/3792495311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.399+0000 7f4ab5353640 1 -- 172.21.15.19:0/3792495311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab010aa20 con 0x7f4ab01080f0 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.400+0000 7f4ab5353640 1 --2- 172.21.15.19:0/3792495311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab010a4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009820 tx=0x7f4aa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8896b1ac187cb75e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.400+0000 7f4aaffff640 1 -- 172.21.15.19:0/3792495311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa8037070 con 0x7f4ab01080f0 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.400+0000 7f4aaffff640 1 -- 172.21.15.19:0/3792495311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4aa8032900 con 0x7f4ab01080f0 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.400+0000 7f4aaffff640 1 -- 172.21.15.19:0/3792495311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa8030850 con 0x7f4ab01080f0 2024-09-19T03:57:27.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.400+0000 7f4ab75b7640 1 -- 172.21.15.19:0/3792495311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 msgr2=0x7f4ab010a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.400+0000 7f4ab75b7640 1 --2- 172.21.15.19:0/3792495311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab010a4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009820 tx=0x7f4aa802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 -- 172.21.15.19:0/3792495311 shutdown_connections 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 --2- 172.21.15.19:0/3792495311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab010a4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 -- 172.21.15.19:0/3792495311 >> 172.21.15.19:0/3792495311 conn(0x7f4ab0101d10 msgr2=0x7f4ab0104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 -- 172.21.15.19:0/3792495311 shutdown_connections 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 -- 172.21.15.19:0/3792495311 wait complete. 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 Processor -- start 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 -- start start 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.401+0000 7f4ab75b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab010bac0 con 0x7f4ab01080f0 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4ab5353640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:27.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4ab5353640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab019d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58996/0 (socket says 172.21.15.19:58996) 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4ab5353640 1 -- 172.21.15.19:0/2414827713 learned_addr learned my addr 172.21.15.19:0/2414827713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4ab5353640 1 -- 172.21.15.19:0/2414827713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab019ddb0 con 0x7f4ab01080f0 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4ab5353640 1 --2- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab019d870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8032ce0 tx=0x7f4aa8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4aae7fc640 1 -- 172.21.15.19:0/2414827713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa8042070 con 0x7f4ab01080f0 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4aae7fc640 1 -- 172.21.15.19:0/2414827713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4aa80308c0 con 0x7f4ab01080f0 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab019e040 con 0x7f4ab01080f0 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4aae7fc640 1 -- 172.21.15.19:0/2414827713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa8037040 con 0x7f4ab01080f0 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.402+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01a0d30 con 0x7f4ab01080f0 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.403+0000 7f4aae7fc640 1 -- 172.21.15.19:0/2414827713 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4aa8045070 con 0x7f4ab01080f0 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.403+0000 7f4aae7fc640 1 --2- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f4a98039200 0x7f4a9803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:27.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.403+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab0108550 con 0x7f4ab01080f0 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.403+0000 7f4aae7fc640 1 -- 172.21.15.19:0/2414827713 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4aa806c450 con 0x7f4ab01080f0 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.403+0000 7f4ab4b52640 1 --2- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f4a98039200 0x7f4a9803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.404+0000 7f4ab4b52640 1 --2- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f4a98039200 0x7f4a9803b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40098c0 tx=0x7f4aa4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.408+0000 7f4aae7fc640 1 -- 172.21.15.19:0/2414827713 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4aa803d430 con 0x7f4ab01080f0 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:22.587+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7f4ab0102f70 con 0x7f4a98039200 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.523+0000 7f4aae7fc640 1 -- 172.21.15.19:0/2414827713 <== mgr.14118 v2:172.21.15.19:6800/2229567042 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4ab0102f70 con 0x7f4a98039200 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.525+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f4a98039200 msgr2=0x7f4a9803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.525+0000 7f4ab75b7640 1 --2- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f4a98039200 0x7f4a9803b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40098c0 tx=0x7f4aa4006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.525+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 msgr2=0x7f4ab019d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:27.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.525+0000 7f4ab75b7640 1 --2- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab019d870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8032ce0 tx=0x7f4aa8032d10 comp rx=0 tx=0).stop 2024-09-19T03:57:27.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.526+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 shutdown_connections 2024-09-19T03:57:27.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.526+0000 7f4ab75b7640 1 --2- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f4a98039200 0x7f4a9803b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:27.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.526+0000 7f4ab75b7640 1 --2- 172.21.15.19:0/2414827713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ab01080f0 0x7f4ab019d870 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:27.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.526+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 >> 172.21.15.19:0/2414827713 conn(0x7f4ab0101d10 msgr2=0x7f4ab0102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:27.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.526+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 shutdown_connections 2024-09-19T03:57:27.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:26.526+0000 7f4ab75b7640 1 -- 172.21.15.19:0/2414827713 wait complete. 2024-09-19T03:57:27.199 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-09-19T03:57:27.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:27 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:27.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:27 smithi019 ceph-mon[30938]: Added host smithi019 2024-09-19T03:57:27.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:27 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.646+0000 7f92f2620640 1 Processor -- start 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.646+0000 7f92f2620640 1 -- start start 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.646+0000 7f92f2620640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.646+0000 7f92f2620640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec1080f0 con 0x7f92ec107720 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.647+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.647+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59002/0 (socket says 172.21.15.19:59002) 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.647+0000 7f92ebfff640 1 -- 172.21.15.19:0/3590170805 learned_addr learned my addr 172.21.15.19:0/3590170805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.647+0000 7f92ebfff640 1 -- 172.21.15.19:0/3590170805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec108920 con 0x7f92ec107720 2024-09-19T03:57:28.672 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.648+0000 7f92ebfff640 1 --2- 172.21.15.19:0/3590170805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec107b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009820 tx=0x7f92dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6ac70658b68f51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.648+0000 7f92eaffd640 1 -- 172.21.15.19:0/3590170805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92dc037070 con 0x7f92ec107720 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.648+0000 7f92eaffd640 1 -- 172.21.15.19:0/3590170805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92dc032900 con 0x7f92ec107720 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.648+0000 7f92eaffd640 1 -- 172.21.15.19:0/3590170805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92dc030850 con 0x7f92ec107720 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 -- 172.21.15.19:0/3590170805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 msgr2=0x7f92ec107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 --2- 172.21.15.19:0/3590170805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec107b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009820 tx=0x7f92dc02a270 comp rx=0 tx=0).stop 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 -- 172.21.15.19:0/3590170805 shutdown_connections 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 --2- 172.21.15.19:0/3590170805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec107b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 -- 172.21.15.19:0/3590170805 >> 172.21.15.19:0/3590170805 conn(0x7f92ec102ed0 msgr2=0x7f92ec1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 -- 172.21.15.19:0/3590170805 shutdown_connections 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 -- 172.21.15.19:0/3590170805 wait complete. 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 Processor -- start 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.649+0000 7f92f2620640 1 -- start start 2024-09-19T03:57:28.673 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92f2620640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92f2620640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec10bcd0 con 0x7f92ec107720 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59014/0 (socket says 172.21.15.19:59014) 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92ebfff640 1 -- 172.21.15.19:0/721589227 learned_addr learned my addr 172.21.15.19:0/721589227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92ebfff640 1 -- 172.21.15.19:0/721589227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec19de90 con 0x7f92ec107720 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92ebfff640 1 --2- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec19d950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0097f0 tx=0x7f92dc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92e97fa640 1 -- 172.21.15.19:0/721589227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92dc042070 con 0x7f92ec107720 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92e97fa640 1 -- 172.21.15.19:0/721589227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92dc032ca0 con 0x7f92ec107720 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec19e120 con 0x7f92ec107720 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92e97fa640 1 -- 172.21.15.19:0/721589227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92dc037040 con 0x7f92ec107720 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.650+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec1a0e10 con 0x7f92ec107720 2024-09-19T03:57:28.674 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.651+0000 7f92e97fa640 1 -- 172.21.15.19:0/721589227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f92dc045070 con 0x7f92ec107720 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.651+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec107ba0 con 0x7f92ec107720 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.651+0000 7f92e97fa640 1 --2- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f92d40391b0 0x7f92d403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.651+0000 7f92e97fa640 1 -- 172.21.15.19:0/721589227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f92dc06c210 con 0x7f92ec107720 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.652+0000 7f92eb7fe640 1 --2- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f92d40391b0 0x7f92d403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.652+0000 7f92eb7fe640 1 --2- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f92d40391b0 0x7f92d403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92e00098c0 tx=0x7f92e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.656+0000 7f92e97fa640 1 -- 172.21.15.19:0/721589227 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92dc045520 con 0x7f92ec107720 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.853+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f92ec104150 con 0x7f92d40391b0 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.858+0000 7f92e97fa640 1 -- 172.21.15.19:0/721589227 <== mgr.14118 v2:172.21.15.19:6800/2229567042 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f92ec104150 con 0x7f92d40391b0 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f92d40391b0 msgr2=0x7f92d403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:28.675 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 --2- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f92d40391b0 0x7f92d403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92e00098c0 tx=0x7f92e0006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 msgr2=0x7f92ec19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 --2- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec19d950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0097f0 tx=0x7f92dc030ec0 comp rx=0 tx=0).stop 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 shutdown_connections 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 --2- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f92d40391b0 0x7f92d403b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 --2- 172.21.15.19:0/721589227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92ec107720 0x7f92ec19d950 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 >> 172.21.15.19:0/721589227 conn(0x7f92ec102ed0 msgr2=0x7f92ec103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 shutdown_connections 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:27.861+0000 7f92f2620640 1 -- 172.21.15.19:0/721589227 wait complete. 2024-09-19T03:57:28.676 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-09-19T03:57:28.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:28 smithi019 ceph-mon[30938]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:28.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:28 smithi019 ceph-mon[30938]: Saving service mon spec with placement count:5 2024-09-19T03:57:28.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:28 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:30.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T03:57:30.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.299+0000 7fd277dc7640 1 Processor -- start 2024-09-19T03:57:30.220 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.299+0000 7fd277dc7640 1 -- start start 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.300+0000 7fd277dc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27010a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.300+0000 7fd277dc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd27007be50 con 0x7fd270108170 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.300+0000 7fd275b63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27010a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.300+0000 7fd275b63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27010a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59026/0 (socket says 172.21.15.19:59026) 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.300+0000 7fd275b63640 1 -- 172.21.15.19:0/2070517781 learned_addr learned my addr 172.21.15.19:0/2070517781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.300+0000 7fd275b63640 1 -- 172.21.15.19:0/2070517781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27010aaa0 con 0x7fd270108170 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.301+0000 7fd275b63640 1 --2- 172.21.15.19:0/2070517781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27010a560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd268009820 tx=0x7fd26802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9cc0b4efedaa8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.301+0000 7fd274b61640 1 -- 172.21.15.19:0/2070517781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd268037070 con 0x7fd270108170 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.301+0000 7fd274b61640 1 -- 172.21.15.19:0/2070517781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd268032900 con 0x7fd270108170 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.301+0000 7fd274b61640 1 -- 172.21.15.19:0/2070517781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd268030850 con 0x7fd270108170 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.302+0000 7fd277dc7640 1 -- 172.21.15.19:0/2070517781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 msgr2=0x7fd27010a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:30.221 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.302+0000 7fd277dc7640 1 --2- 172.21.15.19:0/2070517781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27010a560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd268009820 tx=0x7fd26802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.302+0000 7fd277dc7640 1 -- 172.21.15.19:0/2070517781 shutdown_connections 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.302+0000 7fd277dc7640 1 --2- 172.21.15.19:0/2070517781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27010a560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.302+0000 7fd277dc7640 1 -- 172.21.15.19:0/2070517781 >> 172.21.15.19:0/2070517781 conn(0x7fd270101d10 msgr2=0x7fd270104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.302+0000 7fd277dc7640 1 -- 172.21.15.19:0/2070517781 shutdown_connections 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.302+0000 7fd277dc7640 1 -- 172.21.15.19:0/2070517781 wait complete. 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd277dc7640 1 Processor -- start 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd277dc7640 1 -- start start 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd277dc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd277dc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd27010ba80 con 0x7fd270108170 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd275b63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd275b63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27019b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59030/0 (socket says 172.21.15.19:59030) 2024-09-19T03:57:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd275b63640 1 -- 172.21.15.19:0/1527702051 learned_addr learned my addr 172.21.15.19:0/1527702051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd275b63640 1 -- 172.21.15.19:0/1527702051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27019bbd0 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.303+0000 7fd275b63640 1 --2- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27019b690 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd2680097f0 tx=0x7fd268030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.304+0000 7fd26effd640 1 -- 172.21.15.19:0/1527702051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd268037040 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.304+0000 7fd26effd640 1 -- 172.21.15.19:0/1527702051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd268040070 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.304+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27019be60 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.304+0000 7fd26effd640 1 -- 172.21.15.19:0/1527702051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd26803a390 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.304+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27019eb50 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.305+0000 7fd26effd640 1 -- 172.21.15.19:0/1527702051 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd26803a630 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.305+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2701085d0 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.305+0000 7fd26effd640 1 --2- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fd2580391b0 0x7fd25803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.305+0000 7fd26effd640 1 -- 172.21.15.19:0/1527702051 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd26806c960 con 0x7fd270108170 2024-09-19T03:57:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.305+0000 7fd275362640 1 --2- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fd2580391b0 0x7fd25803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.306+0000 7fd275362640 1 --2- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fd2580391b0 0x7fd25803b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd2640098c0 tx=0x7fd264006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.310+0000 7fd26effd640 1 -- 172.21.15.19:0/1527702051 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd268032ab0 con 0x7fd270108170 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.499+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd270102fd0 con 0x7fd2580391b0 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.504+0000 7fd26effd640 1 -- 172.21.15.19:0/1527702051 <== mgr.14118 v2:172.21.15.19:6800/2229567042 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd270102fd0 con 0x7fd2580391b0 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fd2580391b0 msgr2=0x7fd25803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 --2- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fd2580391b0 0x7fd25803b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd2640098c0 tx=0x7fd264006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 msgr2=0x7fd27019b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 --2- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27019b690 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd2680097f0 tx=0x7fd268030d80 comp rx=0 tx=0).stop 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 shutdown_connections 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 --2- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7fd2580391b0 0x7fd25803b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 --2- 172.21.15.19:0/1527702051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270108170 0x7fd27019b690 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 >> 172.21.15.19:0/1527702051 conn(0x7fd270101d10 msgr2=0x7fd2701028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 shutdown_connections 2024-09-19T03:57:30.225 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:29.506+0000 7fd277dc7640 1 -- 172.21.15.19:0/1527702051 wait complete. 2024-09-19T03:57:30.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:30 smithi019 ceph-mon[30938]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:30.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:30 smithi019 ceph-mon[30938]: Saving service mgr spec with placement count:2 2024-09-19T03:57:30.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:30 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:31.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:31 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:31.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:31 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:31.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:31 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4004414144' entity='client.admin' 2024-09-19T03:57:31.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.019+0000 7f0a6a762640 1 Processor -- start 2024-09-19T03:57:31.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.019+0000 7f0a6a762640 1 -- start start 2024-09-19T03:57:31.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.019+0000 7f0a6a762640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a64106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:31.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.019+0000 7f0a6a762640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64106f40 con 0x7f0a64106570 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.020+0000 7f0a63fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a64106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.020+0000 7f0a63fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a64106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59034/0 (socket says 172.21.15.19:59034) 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.020+0000 7f0a63fff640 1 -- 172.21.15.19:0/4187165206 learned_addr learned my addr 172.21.15.19:0/4187165206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.020+0000 7f0a63fff640 1 -- 172.21.15.19:0/4187165206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a64107770 con 0x7f0a64106570 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.021+0000 7f0a63fff640 1 --2- 172.21.15.19:0/4187165206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a64106970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009820 tx=0x7f0a5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b86b24322412f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.021+0000 7f0a62ffd640 1 -- 172.21.15.19:0/4187165206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a54037070 con 0x7f0a64106570 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.021+0000 7f0a62ffd640 1 -- 172.21.15.19:0/4187165206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a54032920 con 0x7f0a64106570 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.021+0000 7f0a62ffd640 1 -- 172.21.15.19:0/4187165206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a54030810 con 0x7f0a64106570 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.021+0000 7f0a6a762640 1 -- 172.21.15.19:0/4187165206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 msgr2=0x7f0a64106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.021+0000 7f0a6a762640 1 --2- 172.21.15.19:0/4187165206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a64106970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009820 tx=0x7f0a5402a270 comp rx=0 tx=0).stop 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.022+0000 7f0a6a762640 1 -- 172.21.15.19:0/4187165206 shutdown_connections 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.022+0000 7f0a6a762640 1 --2- 172.21.15.19:0/4187165206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a64106970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.022+0000 7f0a6a762640 1 -- 172.21.15.19:0/4187165206 >> 172.21.15.19:0/4187165206 conn(0x7f0a64101d20 msgr2=0x7f0a64104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.022+0000 7f0a6a762640 1 -- 172.21.15.19:0/4187165206 shutdown_connections 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.022+0000 7f0a6a762640 1 -- 172.21.15.19:0/4187165206 wait complete. 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.022+0000 7f0a6a762640 1 Processor -- start 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.022+0000 7f0a6a762640 1 -- start start 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a6a762640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a6419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a6a762640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6410ab20 con 0x7f0a64106570 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a63fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a6419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a63fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a6419b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59050/0 (socket says 172.21.15.19:59050) 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a63fff640 1 -- 172.21.15.19:0/4004414144 learned_addr learned my addr 172.21.15.19:0/4004414144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a63fff640 1 -- 172.21.15.19:0/4004414144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6419bc20 con 0x7f0a64106570 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a63fff640 1 --2- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a6419b6e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0a54030560 tx=0x7f0a54032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:31.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a617fa640 1 -- 172.21.15.19:0/4004414144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a54037040 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a617fa640 1 -- 172.21.15.19:0/4004414144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a54040070 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6419beb0 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a617fa640 1 -- 172.21.15.19:0/4004414144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a5403a740 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.023+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6419eba0 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.024+0000 7f0a617fa640 1 -- 172.21.15.19:0/4004414144 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0a5403a9b0 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.024+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6410aca0 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.025+0000 7f0a617fa640 1 --2- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0a4c0391b0 0x7f0a4c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.025+0000 7f0a617fa640 1 -- 172.21.15.19:0/4004414144 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a5406c850 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.025+0000 7f0a637fe640 1 --2- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0a4c0391b0 0x7f0a4c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.025+0000 7f0a637fe640 1 --2- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0a4c0391b0 0x7f0a4c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a580098c0 tx=0x7f0a58006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.029+0000 7f0a617fa640 1 -- 172.21.15.19:0/4004414144 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0a54041270 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.234+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0a641069f0 con 0x7f0a64106570 2024-09-19T03:57:31.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.239+0000 7f0a617fa640 1 -- 172.21.15.19:0/4004414144 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0a5403ada0 con 0x7f0a64106570 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0a4c0391b0 msgr2=0x7f0a4c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 --2- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0a4c0391b0 0x7f0a4c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a580098c0 tx=0x7f0a58006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 msgr2=0x7f0a6419b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 --2- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a6419b6e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0a54030560 tx=0x7f0a54032670 comp rx=0 tx=0).stop 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 shutdown_connections 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 --2- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f0a4c0391b0 0x7f0a4c03b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 --2- 172.21.15.19:0/4004414144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a64106570 0x7f0a6419b6e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 >> 172.21.15.19:0/4004414144 conn(0x7f0a64101d20 msgr2=0x7f0a64102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 shutdown_connections 2024-09-19T03:57:31.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:31.242+0000 7f0a6a762640 1 -- 172.21.15.19:0/4004414144 wait complete. 2024-09-19T03:57:33.211 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.279+0000 7f697c757640 1 Processor -- start 2024-09-19T03:57:33.211 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.279+0000 7f697c757640 1 -- start start 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.280+0000 7f697c757640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f6974109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.280+0000 7f697c757640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f697410a2b0 con 0x7f69741098e0 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.280+0000 7f697a4f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f6974109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.280+0000 7f697a4f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f6974109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35926/0 (socket says 172.21.15.19:35926) 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.280+0000 7f697a4f3640 1 -- 172.21.15.19:0/3706361320 learned_addr learned my addr 172.21.15.19:0/3706361320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.280+0000 7f697a4f3640 1 -- 172.21.15.19:0/3706361320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697410aae0 con 0x7f69741098e0 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.281+0000 7f697a4f3640 1 --2- 172.21.15.19:0/3706361320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f6974109ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6970009820 tx=0x7f697002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f227ca675c1742f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.281+0000 7f69794f1640 1 -- 172.21.15.19:0/3706361320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6970037070 con 0x7f69741098e0 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.281+0000 7f69794f1640 1 -- 172.21.15.19:0/3706361320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6970032920 con 0x7f69741098e0 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.281+0000 7f69794f1640 1 -- 172.21.15.19:0/3706361320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6970030810 con 0x7f69741098e0 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.281+0000 7f697c757640 1 -- 172.21.15.19:0/3706361320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 msgr2=0x7f6974109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:33.212 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.281+0000 7f697c757640 1 --2- 172.21.15.19:0/3706361320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f6974109ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6970009820 tx=0x7f697002a270 comp rx=0 tx=0).stop 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 -- 172.21.15.19:0/3706361320 shutdown_connections 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 --2- 172.21.15.19:0/3706361320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f6974109ce0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 -- 172.21.15.19:0/3706361320 >> 172.21.15.19:0/3706361320 conn(0x7f697407ceb0 msgr2=0x7f697407d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 -- 172.21.15.19:0/3706361320 shutdown_connections 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 -- 172.21.15.19:0/3706361320 wait complete. 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 Processor -- start 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 -- start start 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f697419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.282+0000 7f697c757640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f697410de90 con 0x7f69741098e0 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f697a4f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f697419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f697a4f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f697419d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35934/0 (socket says 172.21.15.19:35934) 2024-09-19T03:57:33.214 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f697a4f3640 1 -- 172.21.15.19:0/173025760 learned_addr learned my addr 172.21.15.19:0/173025760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:33.214 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f697a4f3640 1 -- 172.21.15.19:0/173025760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697419df00 con 0x7f69741098e0 2024-09-19T03:57:33.214 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f697a4f3640 1 --2- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f697419d9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f69700097f0 tx=0x7f69700327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:33.214 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f696b7fe640 1 -- 172.21.15.19:0/173025760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6970037040 con 0x7f69741098e0 2024-09-19T03:57:33.214 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f696b7fe640 1 -- 172.21.15.19:0/173025760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6970030cc0 con 0x7f69741098e0 2024-09-19T03:57:33.214 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697419e190 con 0x7f69741098e0 2024-09-19T03:57:33.214 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f696b7fe640 1 -- 172.21.15.19:0/173025760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697003a640 con 0x7f69741098e0 2024-09-19T03:57:33.214 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.283+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69741a0e80 con 0x7f69741098e0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.284+0000 7f696b7fe640 1 -- 172.21.15.19:0/173025760 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f697003a940 con 0x7f69741098e0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.284+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697410e010 con 0x7f69741098e0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.284+0000 7f696b7fe640 1 --2- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f69600391b0 0x7f696003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.284+0000 7f696b7fe640 1 -- 172.21.15.19:0/173025760 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f697006c9e0 con 0x7f69741098e0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.284+0000 7f6979cf2640 1 --2- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f69600391b0 0x7f696003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.285+0000 7f6979cf2640 1 --2- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f69600391b0 0x7f696003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f69640098c0 tx=0x7f6964006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.289+0000 7f696b7fe640 1 -- 172.21.15.19:0/173025760 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f697003ba40 con 0x7f69741098e0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.465+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f69741a1050 con 0x7f69741098e0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.467+0000 7f696b7fe640 1 -- 172.21.15.19:0/173025760 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f697003ad80 con 0x7f69741098e0 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f69600391b0 msgr2=0x7f696003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 --2- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f69600391b0 0x7f696003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f69640098c0 tx=0x7f6964006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:33.215 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 msgr2=0x7f697419d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:33.216 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 --2- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f697419d9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f69700097f0 tx=0x7f69700327b0 comp rx=0 tx=0).stop 2024-09-19T03:57:33.216 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 shutdown_connections 2024-09-19T03:57:33.216 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 --2- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f69600391b0 0x7f696003b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:33.216 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 --2- 172.21.15.19:0/173025760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69741098e0 0x7f697419d9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:33.216 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 >> 172.21.15.19:0/173025760 conn(0x7f697407ceb0 msgr2=0x7f69741075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:33.216 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 shutdown_connections 2024-09-19T03:57:33.216 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:32.470+0000 7f697c757640 1 -- 172.21.15.19:0/173025760 wait complete. 2024-09-19T03:57:33.216 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-09-19T03:57:33.784 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:33 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/173025760' entity='client.admin' 2024-09-19T03:57:34.842 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:34 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:34.842 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:34 smithi019 ceph-mon[30938]: from='mgr.14118 172.21.15.19:0/3170962285' entity='mgr.x' 2024-09-19T03:57:34.842 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:34 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3692072853' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T03:57:35.614 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: ignoring --setuser ceph since I am not root 2024-09-19T03:57:35.615 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: ignoring --setgroup ceph since I am not root 2024-09-19T03:57:35.615 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:35.399+0000 7f07499d4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T03:57:35.615 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:35.458+0000 7f07499d4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T03:57:35.615 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:35.613+0000 7f07499d4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T03:57:35.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:35.683+0000 7f07499d4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T03:57:35.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:35.837+0000 7f07499d4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:57:36.247 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:35.984+0000 7f07499d4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T03:57:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.028+0000 7f178091d640 1 Processor -- start 2024-09-19T03:57:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.029+0000 7f178091d640 1 -- start start 2024-09-19T03:57:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.029+0000 7f178091d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c10a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.029+0000 7f178091d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f177c07bed0 con 0x7f177c1080f0 2024-09-19T03:57:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.029+0000 7f177a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c10a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.029+0000 7f177a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c10a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35942/0 (socket says 172.21.15.19:35942) 2024-09-19T03:57:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.029+0000 7f177a59c640 1 -- 172.21.15.19:0/116125268 learned_addr learned my addr 172.21.15.19:0/116125268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.029+0000 7f177a59c640 1 -- 172.21.15.19:0/116125268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c10aa20 con 0x7f177c1080f0 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.030+0000 7f177a59c640 1 --2- 172.21.15.19:0/116125268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c10a4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f176c009820 tx=0x7f176c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb51ae3351c43741 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.030+0000 7f177959a640 1 -- 172.21.15.19:0/116125268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176c037070 con 0x7f177c1080f0 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.030+0000 7f177959a640 1 -- 172.21.15.19:0/116125268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f176c032920 con 0x7f177c1080f0 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.030+0000 7f177959a640 1 -- 172.21.15.19:0/116125268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176c030810 con 0x7f177c1080f0 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.031+0000 7f178091d640 1 -- 172.21.15.19:0/116125268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 msgr2=0x7f177c10a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.031+0000 7f178091d640 1 --2- 172.21.15.19:0/116125268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c10a4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f176c009820 tx=0x7f176c02a270 comp rx=0 tx=0).stop 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.031+0000 7f178091d640 1 -- 172.21.15.19:0/116125268 shutdown_connections 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.031+0000 7f178091d640 1 --2- 172.21.15.19:0/116125268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c10a4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.031+0000 7f178091d640 1 -- 172.21.15.19:0/116125268 >> 172.21.15.19:0/116125268 conn(0x7f177c101d10 msgr2=0x7f177c104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.031+0000 7f178091d640 1 -- 172.21.15.19:0/116125268 shutdown_connections 2024-09-19T03:57:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.031+0000 7f178091d640 1 -- 172.21.15.19:0/116125268 wait complete. 2024-09-19T03:57:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.032+0000 7f178091d640 1 Processor -- start 2024-09-19T03:57:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.032+0000 7f178091d640 1 -- start start 2024-09-19T03:57:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.032+0000 7f178091d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.032+0000 7f178091d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f177c10bb00 con 0x7f177c1080f0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.033+0000 7f177a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.033+0000 7f177a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c19b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35946/0 (socket says 172.21.15.19:35946) 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.033+0000 7f177a59c640 1 -- 172.21.15.19:0/3692072853 learned_addr learned my addr 172.21.15.19:0/3692072853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.033+0000 7f177a59c640 1 -- 172.21.15.19:0/3692072853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c19bbd0 con 0x7f177c1080f0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.033+0000 7f177a59c640 1 --2- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c19b690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f176c0097f0 tx=0x7f176c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.033+0000 7f176b7fe640 1 -- 172.21.15.19:0/3692072853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176c037040 con 0x7f177c1080f0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.034+0000 7f176b7fe640 1 -- 172.21.15.19:0/3692072853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f176c030cc0 con 0x7f177c1080f0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.034+0000 7f176b7fe640 1 -- 172.21.15.19:0/3692072853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176c03a6b0 con 0x7f177c1080f0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.034+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c19be60 con 0x7f177c1080f0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.034+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c19eb50 con 0x7f177c1080f0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.035+0000 7f176b7fe640 1 -- 172.21.15.19:0/3692072853 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f176c03baf0 con 0x7f177c1080f0 2024-09-19T03:57:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.035+0000 7f176b7fe640 1 --2- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f1764039200 0x7f176403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.035+0000 7f176b7fe640 1 -- 172.21.15.19:0/3692072853 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f176c06c7f0 con 0x7f177c1080f0 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.035+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c108550 con 0x7f177c1080f0 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.036+0000 7f1779d9b640 1 --2- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f1764039200 0x7f176403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.036+0000 7f1779d9b640 1 --2- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f1764039200 0x7f176403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f17700098c0 tx=0x7f1770006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.041+0000 7f176b7fe640 1 -- 172.21.15.19:0/3692072853 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f176c0371e0 con 0x7f177c1080f0 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:34.270+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f177c19f000 con 0x7f177c1080f0 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.245+0000 7f176b7fe640 1 -- 172.21.15.19:0/3692072853 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f176c03a850 con 0x7f177c1080f0 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.247+0000 7f176b7fe640 1 -- 172.21.15.19:0/3692072853 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f176c044c20 con 0x7f177c1080f0 2024-09-19T03:57:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.248+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f1764039200 msgr2=0x7f176403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.248+0000 7f178091d640 1 --2- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f1764039200 0x7f176403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f17700098c0 tx=0x7f1770006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.248+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 msgr2=0x7f177c19b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.248+0000 7f178091d640 1 --2- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c19b690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f176c0097f0 tx=0x7f176c0327b0 comp rx=0 tx=0).stop 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.248+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 shutdown_connections 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.248+0000 7f178091d640 1 --2- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f1764039200 0x7f176403b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.248+0000 7f178091d640 1 --2- 172.21.15.19:0/3692072853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f177c1080f0 0x7f177c19b690 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.249+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 >> 172.21.15.19:0/3692072853 conn(0x7f177c101d10 msgr2=0x7f177c1028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.249+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 shutdown_connections 2024-09-19T03:57:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:35.249+0000 7f178091d640 1 -- 172.21.15.19:0/3692072853 wait complete. 2024-09-19T03:57:36.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:36 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3692072853' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T03:57:36.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:36 smithi019 ceph-mon[30938]: mgrmap e9: x(active, since 20s) 2024-09-19T03:57:36.579 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:36.315+0000 7f07499d4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T03:57:36.579 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:36.379+0000 7f07499d4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T03:57:36.579 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:36.577+0000 7f07499d4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T03:57:37.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:37 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1053685893' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T03:57:37.782 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:37.529+0000 7f07499d4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T03:57:37.783 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:37.599+0000 7f07499d4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T03:57:37.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-19T03:57:37.793 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.925+0000 7f5975a93640 1 Processor -- start 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.925+0000 7f5975a93640 1 -- start start 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.926+0000 7f5975a93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.926+0000 7f5975a93640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f597007c8f0 con 0x7f597007dec0 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.926+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.926+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35972/0 (socket says 172.21.15.19:35972) 2024-09-19T03:57:37.794 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.926+0000 7f596f7fe640 1 -- 172.21.15.19:0/3936623508 learned_addr learned my addr 172.21.15.19:0/3936623508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.926+0000 7f596f7fe640 1 -- 172.21.15.19:0/3936623508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597007ca70 con 0x7f597007dec0 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.927+0000 7f596f7fe640 1 --2- 172.21.15.19:0/3936623508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597007c320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5960009820 tx=0x7f596002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0a423a78dbd85f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.927+0000 7f596e7fc640 1 -- 172.21.15.19:0/3936623508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5960037070 con 0x7f597007dec0 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.927+0000 7f596e7fc640 1 -- 172.21.15.19:0/3936623508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5960032920 con 0x7f597007dec0 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.927+0000 7f596e7fc640 1 -- 172.21.15.19:0/3936623508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5960030810 con 0x7f597007dec0 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.928+0000 7f5975a93640 1 -- 172.21.15.19:0/3936623508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 msgr2=0x7f597007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.928+0000 7f5975a93640 1 --2- 172.21.15.19:0/3936623508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597007c320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5960009820 tx=0x7f596002a270 comp rx=0 tx=0).stop 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.928+0000 7f5975a93640 1 -- 172.21.15.19:0/3936623508 shutdown_connections 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.928+0000 7f5975a93640 1 --2- 172.21.15.19:0/3936623508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597007c320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.928+0000 7f5975a93640 1 -- 172.21.15.19:0/3936623508 >> 172.21.15.19:0/3936623508 conn(0x7f5970102e40 msgr2=0x7f5970105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:37.795 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.928+0000 7f5975a93640 1 -- 172.21.15.19:0/3936623508 shutdown_connections 2024-09-19T03:57:37.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.928+0000 7f5975a93640 1 -- 172.21.15.19:0/3936623508 wait complete. 2024-09-19T03:57:37.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.928+0000 7f5975a93640 1 Processor -- start 2024-09-19T03:57:37.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.929+0000 7f5975a93640 1 -- start start 2024-09-19T03:57:37.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.929+0000 7f5975a93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:37.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.929+0000 7f5975a93640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f597010dd80 con 0x7f597007dec0 2024-09-19T03:57:37.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.929+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:37.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.929+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597019d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35986/0 (socket says 172.21.15.19:35986) 2024-09-19T03:57:37.796 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.929+0000 7f596f7fe640 1 -- 172.21.15.19:0/1053685893 learned_addr learned my addr 172.21.15.19:0/1053685893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.929+0000 7f596f7fe640 1 -- 172.21.15.19:0/1053685893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597019de00 con 0x7f597007dec0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.929+0000 7f596f7fe640 1 --2- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597019d8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59600097f0 tx=0x7f59600327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.930+0000 7f596cff9640 1 -- 172.21.15.19:0/1053685893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5960037040 con 0x7f597007dec0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.930+0000 7f596cff9640 1 -- 172.21.15.19:0/1053685893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5960030cc0 con 0x7f597007dec0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.930+0000 7f596cff9640 1 -- 172.21.15.19:0/1053685893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596003a6b0 con 0x7f597007dec0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.930+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597019e090 con 0x7f597007dec0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.930+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59701a0d80 con 0x7f597007dec0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.931+0000 7f596cff9640 1 -- 172.21.15.19:0/1053685893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f596003b3f0 con 0x7f597007dec0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.931+0000 7f596cff9640 1 --2- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5958039260 0x7f595803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.931+0000 7f596cff9640 1 -- 172.21.15.19:0/1053685893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f596006c7b0 con 0x7f597007dec0 2024-09-19T03:57:37.797 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.931+0000 7f596effd640 1 -- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5958039260 msgr2=0x7f595803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2229567042 2024-09-19T03:57:37.798 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.931+0000 7f596effd640 1 --2- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5958039260 0x7f595803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T03:57:37.798 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.931+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597007c3a0 con 0x7f597007dec0 2024-09-19T03:57:37.798 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:36.936+0000 7f596cff9640 1 -- 172.21.15.19:0/1053685893 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f596003a850 con 0x7f597007dec0 2024-09-19T03:57:37.798 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.132+0000 7f596effd640 1 -- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5958039260 msgr2=0x7f595803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2229567042 2024-09-19T03:57:37.798 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.132+0000 7f596effd640 1 --2- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5958039260 0x7f595803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T03:57:37.798 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.159+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f59701a11a0 con 0x7f597007dec0 2024-09-19T03:57:37.798 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.159+0000 7f596cff9640 1 -- 172.21.15.19:0/1053685893 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5960049cc0 con 0x7f597007dec0 2024-09-19T03:57:37.798 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5958039260 msgr2=0x7f595803b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T03:57:37.799 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 --2- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5958039260 0x7f595803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:37.799 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 msgr2=0x7f597019d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:37.799 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 --2- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597019d8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59600097f0 tx=0x7f59600327b0 comp rx=0 tx=0).stop 2024-09-19T03:57:37.799 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 shutdown_connections 2024-09-19T03:57:37.799 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 --2- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5958039260 0x7f595803b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:37.799 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 --2- 172.21.15.19:0/1053685893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597007dec0 0x7f597019d8c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:37.800 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 >> 172.21.15.19:0/1053685893 conn(0x7f5970102e40 msgr2=0x7f59701039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:37.800 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 shutdown_connections 2024-09-19T03:57:37.800 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:37.163+0000 7f5975a93640 1 -- 172.21.15.19:0/1053685893 wait complete. 2024-09-19T03:57:37.800 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-09-19T03:57:37.800 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-09-19T03:57:38.634 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:38.362+0000 7f07499d4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T03:57:38.634 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:38.484+0000 7f07499d4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T03:57:38.924 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:38.633+0000 7f07499d4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:57:38.924 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:38.697+0000 7f07499d4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T03:57:38.925 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:38.852+0000 7f07499d4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T03:57:38.925 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:38.923+0000 7f07499d4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T03:57:39.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:38.985+0000 7f07499d4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T03:57:39.181 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:39.151+0000 7f07499d4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T03:57:39.499 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:39.218+0000 7f07499d4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T03:57:39.499 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:39.288+0000 7f07499d4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T03:57:39.499 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:39.374+0000 7f07499d4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T03:57:39.499 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:39.436+0000 7f07499d4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T03:57:39.499 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:39.497+0000 7f07499d4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T03:57:40.586 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:40.324+0000 7f07499d4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T03:57:40.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: Active manager daemon x restarted 2024-09-19T03:57:40.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: Activating manager daemon x 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: mgrmap e10: x(active, starting, since 0.00317281s) 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: Manager daemon x is now available 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T03:57:40.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T03:57:42.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-19T03:57:42.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T03:57:42.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.215+0000 7f556384e640 1 Processor -- start 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.215+0000 7f556384e640 1 -- start start 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.216+0000 7f556384e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.216+0000 7f556384e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f555c07be80 con 0x7f555c108180 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.216+0000 7f55615ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.216+0000 7f55615ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35996/0 (socket says 172.21.15.19:35996) 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.216+0000 7f55615ea640 1 -- 172.21.15.19:0/3162426318 learned_addr learned my addr 172.21.15.19:0/3162426318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.216+0000 7f55615ea640 1 -- 172.21.15.19:0/3162426318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c10aab0 con 0x7f555c108180 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.217+0000 7f55615ea640 1 --2- 172.21.15.19:0/3162426318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c10a570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5558009820 tx=0x7f555802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d94ab22e0a6c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.217+0000 7f5553fff640 1 -- 172.21.15.19:0/3162426318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558037070 con 0x7f555c108180 2024-09-19T03:57:42.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.217+0000 7f5553fff640 1 -- 172.21.15.19:0/3162426318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5558032920 con 0x7f555c108180 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.217+0000 7f5553fff640 1 -- 172.21.15.19:0/3162426318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558030810 con 0x7f555c108180 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.218+0000 7f556384e640 1 -- 172.21.15.19:0/3162426318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 msgr2=0x7f555c10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.218+0000 7f556384e640 1 --2- 172.21.15.19:0/3162426318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c10a570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5558009820 tx=0x7f555802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.218+0000 7f556384e640 1 -- 172.21.15.19:0/3162426318 shutdown_connections 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.218+0000 7f556384e640 1 --2- 172.21.15.19:0/3162426318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c10a570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.218+0000 7f556384e640 1 -- 172.21.15.19:0/3162426318 >> 172.21.15.19:0/3162426318 conn(0x7f555c101da0 msgr2=0x7f555c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.218+0000 7f556384e640 1 -- 172.21.15.19:0/3162426318 shutdown_connections 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.218+0000 7f556384e640 1 -- 172.21.15.19:0/3162426318 wait complete. 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f556384e640 1 Processor -- start 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f556384e640 1 -- start start 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f556384e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c107a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f556384e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f555c10bb30 con 0x7f555c108180 2024-09-19T03:57:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f55615ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f55615ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36000/0 (socket says 172.21.15.19:36000) 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f55615ea640 1 -- 172.21.15.19:0/1921705137 learned_addr learned my addr 172.21.15.19:0/1921705137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f55615ea640 1 -- 172.21.15.19:0/1921705137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c107f40 con 0x7f555c108180 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.219+0000 7f55615ea640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c107a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5558030420 tx=0x7f5558032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.220+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558037040 con 0x7f555c108180 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.220+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5558040070 con 0x7f555c108180 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.220+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555c106130 con 0x7f555c108180 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.220+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f555803a6e0 con 0x7f555c108180 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.220+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555c106670 con 0x7f555c108180 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.221+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f555803a990 con 0x7f555c108180 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.221+0000 7f55527fc640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 0x7f554803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.221+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 --> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f554803be30 con 0x7f5548039260 2024-09-19T03:57:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.221+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f555806c9d0 con 0x7f555c108180 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.221+0000 7f5560de9640 1 -- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 msgr2=0x7f554803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2229567042 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.221+0000 7f5560de9640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 0x7f554803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.422+0000 7f5560de9640 1 -- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 msgr2=0x7f554803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2229567042 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.422+0000 7f5560de9640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 0x7f554803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.823+0000 7f5560de9640 1 -- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 msgr2=0x7f554803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2229567042 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:38.823+0000 7f5560de9640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 0x7f554803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:39.624+0000 7f5560de9640 1 -- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 msgr2=0x7f554803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2229567042 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:39.624+0000 7f5560de9640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 0x7f554803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:40.327+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f555806b480 con 0x7f555c108180 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:40.328+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 msgr2=0x7f554803b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:40.328+0000 7f55527fc640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 0x7f554803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:42.247 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.329+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f5558045980 con 0x7f555c108180 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.330+0000 7f55527fc640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f554803cd20 0x7f554803f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.330+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5558038e50 con 0x7f554803cd20 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.330+0000 7f5560de9640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f554803cd20 0x7f554803f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.331+0000 7f5560de9640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f554803cd20 0x7f554803f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f554c003980 tx=0x7f554c0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.331+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5558038e50 con 0x7f554803cd20 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.338+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f555c1085e0 con 0x7f554803cd20 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.338+0000 7f55527fc640 1 -- 172.21.15.19:0/1921705137 <== mgr.14150 v2:172.21.15.19:6800/2387193681 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f555c1085e0 con 0x7f554803cd20 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f554803cd20 msgr2=0x7f554803f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f554803cd20 0x7f554803f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f554c003980 tx=0x7f554c0092b0 comp rx=0 tx=0).stop 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 msgr2=0x7f555c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c107a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5558030420 tx=0x7f5558032670 comp rx=0 tx=0).stop 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 shutdown_connections 2024-09-19T03:57:42.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f554803cd20 0x7f554803f110 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:42.249 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:6800/2229567042,v1:172.21.15.19:6801/2229567042] conn(0x7f5548039260 0x7f554803b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:42.249 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 --2- 172.21.15.19:0/1921705137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c108180 0x7f555c107a00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:42.249 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 >> 172.21.15.19:0/1921705137 conn(0x7f555c101da0 msgr2=0x7f555c1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:42.249 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 shutdown_connections 2024-09-19T03:57:42.249 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:41.339+0000 7f556384e640 1 -- 172.21.15.19:0/1921705137 wait complete. 2024-09-19T03:57:42.249 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-09-19T03:57:42.249 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-09-19T03:57:42.331 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:42.057+0000 7f06af49d640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:03:57:42] ENGINE Error in HTTPServer.serve 2024-09-19T03:57:42.331 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: Traceback (most recent call last): 2024-09-19T03:57:42.331 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T03:57:42.331 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: self._connections.run(self.expiration_interval) 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: self._run(expiration_interval) 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: s = self.context.wrap_socket( 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: return self.sslsocket_class._create( 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T03:57:42.332 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: self.do_handshake() 2024-09-19T03:57:42.333 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T03:57:42.333 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: self._sslobj.do_handshake() 2024-09-19T03:57:42.333 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T03:57:42.333 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T03:57:42.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-mon[30938]: mgrmap e11: x(active, since 1.00598s) 2024-09-19T03:57:42.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:41] ENGINE Bus STARTING 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:41] ENGINE Serving on http://172.21.15.19:8765 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:42] ENGINE Serving on https://172.21.15.19:7150 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:42] ENGINE Bus STARTED 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:42 smithi019 ceph-mon[30938]: [19/Sep/2024:03:57:42] ENGINE Error in HTTPServer.serve 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout: Traceback (most recent call last): 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._connections.run(self.expiration_interval) 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._run(expiration_interval) 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T03:57:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: s = self.context.wrap_socket( 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: return self.sslsocket_class._create( 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: self.do_handshake() 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._sslobj.do_handshake() 2024-09-19T03:57:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T03:57:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:43 smithi019 ceph-mon[30938]: mgrmap e12: x(active, since 2s) 2024-09-19T03:57:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:43 smithi019 ceph-mon[30938]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:43.978 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T03:57:43.978 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.701+0000 7f7a54276640 1 Processor -- start 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.701+0000 7f7a54276640 1 -- start start 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.701+0000 7f7a54276640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c10a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.701+0000 7f7a54276640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c07be10 con 0x7f7a4c1081b0 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.701+0000 7f7a52012640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c10a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.701+0000 7f7a52012640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c10a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43876/0 (socket says 172.21.15.19:43876) 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.701+0000 7f7a52012640 1 -- 172.21.15.19:0/3441955751 learned_addr learned my addr 172.21.15.19:0/3441955751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.701+0000 7f7a52012640 1 -- 172.21.15.19:0/3441955751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c10aae0 con 0x7f7a4c1081b0 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.702+0000 7f7a52012640 1 --2- 172.21.15.19:0/3441955751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c10a5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009820 tx=0x7f7a4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f87c204f5a3f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.702+0000 7f7a51010640 1 -- 172.21.15.19:0/3441955751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a48037070 con 0x7f7a4c1081b0 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.702+0000 7f7a51010640 1 -- 172.21.15.19:0/3441955751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a48032920 con 0x7f7a4c1081b0 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.702+0000 7f7a51010640 1 -- 172.21.15.19:0/3441955751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a48030810 con 0x7f7a4c1081b0 2024-09-19T03:57:43.979 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.703+0000 7f7a54276640 1 -- 172.21.15.19:0/3441955751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 msgr2=0x7f7a4c10a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.703+0000 7f7a54276640 1 --2- 172.21.15.19:0/3441955751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c10a5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009820 tx=0x7f7a4802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.703+0000 7f7a54276640 1 -- 172.21.15.19:0/3441955751 shutdown_connections 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.703+0000 7f7a54276640 1 --2- 172.21.15.19:0/3441955751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c10a5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.703+0000 7f7a54276640 1 -- 172.21.15.19:0/3441955751 >> 172.21.15.19:0/3441955751 conn(0x7f7a4c101d10 msgr2=0x7f7a4c104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.703+0000 7f7a54276640 1 -- 172.21.15.19:0/3441955751 shutdown_connections 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.703+0000 7f7a54276640 1 -- 172.21.15.19:0/3441955751 wait complete. 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.704+0000 7f7a54276640 1 Processor -- start 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.704+0000 7f7a54276640 1 -- start start 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.704+0000 7f7a54276640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.704+0000 7f7a54276640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c10bb00 con 0x7f7a4c1081b0 2024-09-19T03:57:43.980 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.704+0000 7f7a52012640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.704+0000 7f7a52012640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c19b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43890/0 (socket says 172.21.15.19:43890) 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.704+0000 7f7a52012640 1 -- 172.21.15.19:0/3563425570 learned_addr learned my addr 172.21.15.19:0/3563425570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.705+0000 7f7a52012640 1 -- 172.21.15.19:0/3563425570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c19bbd0 con 0x7f7a4c1081b0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.705+0000 7f7a52012640 1 --2- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c19b690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a48030380 tx=0x7f7a48032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.705+0000 7f7a3f7fe640 1 -- 172.21.15.19:0/3563425570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a48037040 con 0x7f7a4c1081b0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.705+0000 7f7a3f7fe640 1 -- 172.21.15.19:0/3563425570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a48040070 con 0x7f7a4c1081b0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.705+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c19be60 con 0x7f7a4c1081b0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.705+0000 7f7a3f7fe640 1 -- 172.21.15.19:0/3563425570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a4803a7b0 con 0x7f7a4c1081b0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.705+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c19eb50 con 0x7f7a4c1081b0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.706+0000 7f7a3f7fe640 1 -- 172.21.15.19:0/3563425570 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f7a4803aa30 con 0x7f7a4c1081b0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.706+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4c108610 con 0x7f7a4c1081b0 2024-09-19T03:57:43.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.706+0000 7f7a3f7fe640 1 --2- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7a38039200 0x7f7a3803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.706+0000 7f7a3f7fe640 1 -- 172.21.15.19:0/3563425570 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a4806cb40 con 0x7f7a4c1081b0 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.706+0000 7f7a51811640 1 --2- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7a38039200 0x7f7a3803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.707+0000 7f7a51811640 1 --2- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7a38039200 0x7f7a3803b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7a400098c0 tx=0x7f7a40006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.711+0000 7f7a3f7fe640 1 -- 172.21.15.19:0/3563425570 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7a48044c70 con 0x7f7a4c1081b0 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:42.902+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f7a4c103000 con 0x7f7a38039200 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.045+0000 7f7a3f7fe640 1 -- 172.21.15.19:0/3563425570 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f7a4c103000 con 0x7f7a38039200 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7a38039200 msgr2=0x7f7a3803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 --2- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7a38039200 0x7f7a3803b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7a400098c0 tx=0x7f7a40006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 msgr2=0x7f7a4c19b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 --2- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c19b690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a48030380 tx=0x7f7a48032670 comp rx=0 tx=0).stop 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 shutdown_connections 2024-09-19T03:57:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 --2- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7a38039200 0x7f7a3803b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 --2- 172.21.15.19:0/3563425570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4c1081b0 0x7f7a4c19b690 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 >> 172.21.15.19:0/3563425570 conn(0x7f7a4c101d10 msgr2=0x7f7a4c1028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 shutdown_connections 2024-09-19T03:57:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:43.047+0000 7f7a54276640 1 -- 172.21.15.19:0/3563425570 wait complete. 2024-09-19T03:57:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-09-19T03:57:45.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xhEcSi.Pljal.jPHdEjZy.s93ri4Kx3KW7MKwNCjrNhEeJrw.6dPO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726718264, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T03:57:45.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.402+0000 7f37dcb58640 1 Processor -- start 2024-09-19T03:57:45.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.402+0000 7f37dcb58640 1 -- start start 2024-09-19T03:57:45.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.402+0000 7f37dcb58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d810a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:45.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.402+0000 7f37dcb58640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37d807be50 con 0x7f37d81081c0 2024-09-19T03:57:45.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.402+0000 7f37d659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d810a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:45.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.402+0000 7f37d659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d810a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43898/0 (socket says 172.21.15.19:43898) 2024-09-19T03:57:45.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.402+0000 7f37d659c640 1 -- 172.21.15.19:0/825556084 learned_addr learned my addr 172.21.15.19:0/825556084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:45.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.403+0000 7f37d659c640 1 -- 172.21.15.19:0/825556084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d810aaf0 con 0x7f37d81081c0 2024-09-19T03:57:45.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.403+0000 7f37d659c640 1 --2- 172.21.15.19:0/825556084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d810a5b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009820 tx=0x7f37c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7689648c948133 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:45.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.403+0000 7f37d5d9b640 1 -- 172.21.15.19:0/825556084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37c8037070 con 0x7f37d81081c0 2024-09-19T03:57:45.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.403+0000 7f37d5d9b640 1 -- 172.21.15.19:0/825556084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c8032920 con 0x7f37d81081c0 2024-09-19T03:57:45.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.404+0000 7f37d5d9b640 1 -- 172.21.15.19:0/825556084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37c8030810 con 0x7f37d81081c0 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.404+0000 7f37dcb58640 1 -- 172.21.15.19:0/825556084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 msgr2=0x7f37d810a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.404+0000 7f37dcb58640 1 --2- 172.21.15.19:0/825556084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d810a5b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009820 tx=0x7f37c802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.404+0000 7f37dcb58640 1 -- 172.21.15.19:0/825556084 shutdown_connections 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.404+0000 7f37dcb58640 1 --2- 172.21.15.19:0/825556084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d810a5b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.404+0000 7f37dcb58640 1 -- 172.21.15.19:0/825556084 >> 172.21.15.19:0/825556084 conn(0x7f37d8101da0 msgr2=0x7f37d81041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.404+0000 7f37dcb58640 1 -- 172.21.15.19:0/825556084 shutdown_connections 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.404+0000 7f37dcb58640 1 -- 172.21.15.19:0/825556084 wait complete. 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.405+0000 7f37dcb58640 1 Processor -- start 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.405+0000 7f37dcb58640 1 -- start start 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.405+0000 7f37dcb58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d8107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.405+0000 7f37dcb58640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810bb10 con 0x7f37d81081c0 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.405+0000 7f37d659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d8107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:45.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.405+0000 7f37d659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d8107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43904/0 (socket says 172.21.15.19:43904) 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.405+0000 7f37d659c640 1 -- 172.21.15.19:0/3979386799 learned_addr learned my addr 172.21.15.19:0/3979386799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.406+0000 7f37d659c640 1 -- 172.21.15.19:0/3979386799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d8107f80 con 0x7f37d81081c0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.406+0000 7f37d659c640 1 --2- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d8107a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f37c8030380 tx=0x7f37c8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.406+0000 7f37cf7fe640 1 -- 172.21.15.19:0/3979386799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37c8037040 con 0x7f37d81081c0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.406+0000 7f37cf7fe640 1 -- 172.21.15.19:0/3979386799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c8040070 con 0x7f37d81081c0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.406+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d8106170 con 0x7f37d81081c0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.406+0000 7f37cf7fe640 1 -- 172.21.15.19:0/3979386799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37c803a7b0 con 0x7f37d81081c0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.406+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81066b0 con 0x7f37d81081c0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.407+0000 7f37cf7fe640 1 -- 172.21.15.19:0/3979386799 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f37c803bc00 con 0x7f37d81081c0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.407+0000 7f37cf7fe640 1 --2- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f37c4039260 0x7f37c403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.407+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d8108620 con 0x7f37d81081c0 2024-09-19T03:57:45.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.407+0000 7f37cf7fe640 1 -- 172.21.15.19:0/3979386799 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f37c806cac0 con 0x7f37d81081c0 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.407+0000 7f37cffff640 1 --2- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f37c4039260 0x7f37c403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.408+0000 7f37cffff640 1 --2- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f37c4039260 0x7f37c403b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37c00098c0 tx=0x7f37c0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.412+0000 7f37cf7fe640 1 -- 172.21.15.19:0/3979386799 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f37c8040220 con 0x7f37d81081c0 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.604+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f37d8106f90 con 0x7f37c4039260 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.853+0000 7f37cf7fe640 1 -- 172.21.15.19:0/3979386799 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f37d8106f90 con 0x7f37c4039260 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.855+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f37c4039260 msgr2=0x7f37c403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.855+0000 7f37dcb58640 1 --2- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f37c4039260 0x7f37c403b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37c00098c0 tx=0x7f37c0006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.855+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 msgr2=0x7f37d8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.855+0000 7f37dcb58640 1 --2- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d8107a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f37c8030380 tx=0x7f37c8032670 comp rx=0 tx=0).stop 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.855+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 shutdown_connections 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.855+0000 7f37dcb58640 1 --2- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f37c4039260 0x7f37c403b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:45.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.855+0000 7f37dcb58640 1 --2- 172.21.15.19:0/3979386799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d81081c0 0x7f37d8107a40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:45.457 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.855+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 >> 172.21.15.19:0/3979386799 conn(0x7f37d8101da0 msgr2=0x7f37d8102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:45.457 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.856+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 shutdown_connections 2024-09-19T03:57:45.457 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:44.856+0000 7f37dcb58640 1 -- 172.21.15.19:0/3979386799 wait complete. 2024-09-19T03:57:45.457 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-09-19T03:57:46.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:45 smithi019 ceph-mon[30938]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:46.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:45 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:46.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T03:57:46.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.885+0000 7f3509f6d640 1 Processor -- start 2024-09-19T03:57:46.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.885+0000 7f3509f6d640 1 -- start start 2024-09-19T03:57:46.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.885+0000 7f3509f6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f3504107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:46.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.885+0000 7f3509f6d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35041080b0 con 0x7f35041076e0 2024-09-19T03:57:46.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.885+0000 7f35037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f3504107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.885+0000 7f35037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f3504107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43918/0 (socket says 172.21.15.19:43918) 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.885+0000 7f35037fe640 1 -- 172.21.15.19:0/63101259 learned_addr learned my addr 172.21.15.19:0/63101259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.885+0000 7f35037fe640 1 -- 172.21.15.19:0/63101259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35041088e0 con 0x7f35041076e0 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.886+0000 7f35037fe640 1 --2- 172.21.15.19:0/63101259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f3504107ae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009820 tx=0x7f34f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d697db21bf480e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.886+0000 7f35027fc640 1 -- 172.21.15.19:0/63101259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34f4037070 con 0x7f35041076e0 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.886+0000 7f35027fc640 1 -- 172.21.15.19:0/63101259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f4032920 con 0x7f35041076e0 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.886+0000 7f35027fc640 1 -- 172.21.15.19:0/63101259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34f4030810 con 0x7f35041076e0 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.887+0000 7f3509f6d640 1 -- 172.21.15.19:0/63101259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 msgr2=0x7f3504107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.887+0000 7f3509f6d640 1 --2- 172.21.15.19:0/63101259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f3504107ae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009820 tx=0x7f34f402a270 comp rx=0 tx=0).stop 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.887+0000 7f3509f6d640 1 -- 172.21.15.19:0/63101259 shutdown_connections 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.887+0000 7f3509f6d640 1 --2- 172.21.15.19:0/63101259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f3504107ae0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:46.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.887+0000 7f3509f6d640 1 -- 172.21.15.19:0/63101259 >> 172.21.15.19:0/63101259 conn(0x7f3504102e90 msgr2=0x7f35041052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.887+0000 7f3509f6d640 1 -- 172.21.15.19:0/63101259 shutdown_connections 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.887+0000 7f3509f6d640 1 -- 172.21.15.19:0/63101259 wait complete. 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.888+0000 7f3509f6d640 1 Processor -- start 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.888+0000 7f3509f6d640 1 -- start start 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.888+0000 7f3509f6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f350419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.888+0000 7f3509f6d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f350410bc90 con 0x7f35041076e0 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.888+0000 7f35037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f350419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.888+0000 7f35037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f350419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43934/0 (socket says 172.21.15.19:43934) 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.888+0000 7f35037fe640 1 -- 172.21.15.19:0/4243252580 learned_addr learned my addr 172.21.15.19:0/4243252580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.888+0000 7f35037fe640 1 -- 172.21.15.19:0/4243252580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350419bca0 con 0x7f35041076e0 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.889+0000 7f35037fe640 1 --2- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f350419b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097f0 tx=0x7f34f40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.889+0000 7f3500ff9640 1 -- 172.21.15.19:0/4243252580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34f4037040 con 0x7f35041076e0 2024-09-19T03:57:46.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.889+0000 7f3500ff9640 1 -- 172.21.15.19:0/4243252580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f4030c40 con 0x7f35041076e0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.889+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350419bf30 con 0x7f35041076e0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.889+0000 7f3500ff9640 1 -- 172.21.15.19:0/4243252580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34f403a5b0 con 0x7f35041076e0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.889+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350419ec20 con 0x7f35041076e0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.890+0000 7f3500ff9640 1 -- 172.21.15.19:0/4243252580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f34f403a7a0 con 0x7f35041076e0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.890+0000 7f3500ff9640 1 --2- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34ec039260 0x7f34ec03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.890+0000 7f3500ff9640 1 -- 172.21.15.19:0/4243252580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f34f403b3f0 con 0x7f35041076e0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.890+0000 7f3502ffd640 1 --2- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34ec039260 0x7f34ec03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.890+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f350410be10 con 0x7f35041076e0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.890+0000 7f3502ffd640 1 --2- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34ec039260 0x7f34ec03b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f34f80098c0 tx=0x7f34f8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:45.895+0000 7f3500ff9640 1 -- 172.21.15.19:0/4243252580 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34f4030df0 con 0x7f35041076e0 2024-09-19T03:57:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.069+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f350419f040 con 0x7f35041076e0 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.070+0000 7f3500ff9640 1 -- 172.21.15.19:0/4243252580 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f34f4031c00 con 0x7f35041076e0 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34ec039260 msgr2=0x7f34ec03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 --2- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34ec039260 0x7f34ec03b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f34f80098c0 tx=0x7f34f8006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 msgr2=0x7f350419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 --2- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f350419b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097f0 tx=0x7f34f40327b0 comp rx=0 tx=0).stop 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 shutdown_connections 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 --2- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34ec039260 0x7f34ec03b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 --2- 172.21.15.19:0/4243252580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041076e0 0x7f350419b760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 >> 172.21.15.19:0/4243252580 conn(0x7f3504102e90 msgr2=0x7f3504103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 shutdown_connections 2024-09-19T03:57:46.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:46.072+0000 7f3509f6d640 1 -- 172.21.15.19:0/4243252580 wait complete. 2024-09-19T03:57:46.626 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T03:57:46.626 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-09-19T03:57:46.634 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T03:57:46.634 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-19T03:57:46.634 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-09-19T03:57:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T03:57:46.636 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-09-19T03:57:46.637 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:46.637 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019.front.sepia.ceph.com:8443/ 2024-09-19T03:57:46.637 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-09-19T03:57:46.637 INFO:teuthology.orchestra.run.smithi019.stdout: Password: xmxathpctc 2024-09-19T03:57:46.637 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:46.637 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config directory 2024-09-19T03:57:46.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:46 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4243252580' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T03:57:47.855 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.044+0000 7fe30cfe1640 1 Processor -- start 2024-09-19T03:57:47.855 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.044+0000 7fe30cfe1640 1 -- start start 2024-09-19T03:57:47.855 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.044+0000 7fe30cfe1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30810a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.044+0000 7fe30cfe1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe30807be90 con 0x7fe308108180 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.045+0000 7fe30659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30810a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.045+0000 7fe30659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30810a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43936/0 (socket says 172.21.15.19:43936) 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.045+0000 7fe30659c640 1 -- 172.21.15.19:0/495800206 learned_addr learned my addr 172.21.15.19:0/495800206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.045+0000 7fe30659c640 1 -- 172.21.15.19:0/495800206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30810aab0 con 0x7fe308108180 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30659c640 1 --2- 172.21.15.19:0/495800206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30810a570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009820 tx=0x7fe2f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbcc9fc27efaf3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30559a640 1 -- 172.21.15.19:0/495800206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2f8037070 con 0x7fe308108180 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30559a640 1 -- 172.21.15.19:0/495800206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f8032920 con 0x7fe308108180 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30559a640 1 -- 172.21.15.19:0/495800206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2f8030810 con 0x7fe308108180 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30cfe1640 1 -- 172.21.15.19:0/495800206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 msgr2=0x7fe30810a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:47.856 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30cfe1640 1 --2- 172.21.15.19:0/495800206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30810a570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009820 tx=0x7fe2f802a270 comp rx=0 tx=0).stop 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30cfe1640 1 -- 172.21.15.19:0/495800206 shutdown_connections 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30cfe1640 1 --2- 172.21.15.19:0/495800206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30810a570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.046+0000 7fe30cfe1640 1 -- 172.21.15.19:0/495800206 >> 172.21.15.19:0/495800206 conn(0x7fe308101da0 msgr2=0x7fe3081041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.047+0000 7fe30cfe1640 1 -- 172.21.15.19:0/495800206 shutdown_connections 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.047+0000 7fe30cfe1640 1 -- 172.21.15.19:0/495800206 wait complete. 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.047+0000 7fe30cfe1640 1 Processor -- start 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.047+0000 7fe30cfe1640 1 -- start start 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.047+0000 7fe30cfe1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.047+0000 7fe30cfe1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe30810bb30 con 0x7fe308108180 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe30659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe30659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30819b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43946/0 (socket says 172.21.15.19:43946) 2024-09-19T03:57:47.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe30659c640 1 -- 172.21.15.19:0/3999475656 learned_addr learned my addr 172.21.15.19:0/3999475656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe30659c640 1 -- 172.21.15.19:0/3999475656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30819bbf0 con 0x7fe308108180 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe30659c640 1 --2- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30819b6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80097f0 tx=0x7fe2f80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe2f77fe640 1 -- 172.21.15.19:0/3999475656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2f8037040 con 0x7fe308108180 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe2f77fe640 1 -- 172.21.15.19:0/3999475656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f8030cc0 con 0x7fe308108180 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe2f77fe640 1 -- 172.21.15.19:0/3999475656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2f803a6b0 con 0x7fe308108180 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30819be80 con 0x7fe308108180 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.048+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe30819eb70 con 0x7fe308108180 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.049+0000 7fe2f77fe640 1 -- 172.21.15.19:0/3999475656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fe2f803a850 con 0x7fe308108180 2024-09-19T03:57:47.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.049+0000 7fe2f77fe640 1 --2- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe2f0039300 0x7fe2f003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.049+0000 7fe2f77fe640 1 -- 172.21.15.19:0/3999475656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe2f806c940 con 0x7fe308108180 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.049+0000 7fe305d9b640 1 --2- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe2f0039300 0x7fe2f003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.049+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3081085e0 con 0x7fe308108180 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.050+0000 7fe305d9b640 1 --2- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe2f0039300 0x7fe2f003b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0098c0 tx=0x7fe2fc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.054+0000 7fe2f77fe640 1 -- 172.21.15.19:0/3999475656 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe2f8039760 con 0x7fe308108180 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.321+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe30819ef90 con 0x7fe308108180 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.323+0000 7fe2f77fe640 1 -- 172.21.15.19:0/3999475656 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe2f80371e0 con 0x7fe308108180 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe2f0039300 msgr2=0x7fe2f003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 --2- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe2f0039300 0x7fe2f003b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0098c0 tx=0x7fe2fc006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 msgr2=0x7fe30819b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:47.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 --2- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30819b6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80097f0 tx=0x7fe2f80327b0 comp rx=0 tx=0).stop 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 shutdown_connections 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 --2- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe2f0039300 0x7fe2f003b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 --2- 172.21.15.19:0/3999475656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308108180 0x7fe30819b6b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 >> 172.21.15.19:0/3999475656 conn(0x7fe308101da0 msgr2=0x7fe308102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 shutdown_connections 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-19T03:57:47.326+0000 7fe30cfe1640 1 -- 172.21.15.19:0/3999475656 wait complete. 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T03:57:47.860 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-09-19T03:57:47.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T03:57:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:47.862 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-09-19T03:57:47.879 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:47 smithi019 ceph-mon[30938]: mgrmap e13: x(active, since 6s) 2024-09-19T03:57:47.879 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:47 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3999475656' entity='client.admin' 2024-09-19T03:57:47.893 INFO:tasks.cephadm:Fetching config... 2024-09-19T03:57:47.894 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:57:47.894 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T03:57:47.910 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T03:57:47.910 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:57:47.910 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T03:57:47.966 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T03:57:47.967 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:57:47.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T03:57:48.033 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T03:57:48.033 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:57:48.033 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T03:57:48.090 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T03:57:48.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu0BwHdO5ZVgfJ6EjY87o5xBngC73LF1kcfsekz1IrsZC4kGdwLn1WRVIPGzDR2znF3uBJDbj78Sncc7xVFKjIcnMqZYpwLXTE9DXA02s/il1El/enDUN6lwvtOQ1eYdO+wjaEbO5lVdZnjczeW5j7XWQ+v2TZo5YB6Tf8SS6hsuNHNASZJgpj/rP40+W2WtYdsuBaEhherzP1cdJrmU8mJhek5BAQo3h9PHQ1vm8PZq+Ca4EnnWMQJhpeafSfbPmAD1kqbuwhDPucyU820VyG7znzaTq3Bxlyf9PcwLvvZQGye9V3B/fxuy/cNZrHMDq7gRxIYkb8DPffmnpx3zkOI9Djqe4ynDWwLhuRj1UNBnzyMyhBi8Z/kY934l1rrzZUe4lP0NzSpXxunJrgvGHMb2lXL6sHr7vdHbSsvctdhwqtVCD+tae35FWhh7zmGft7MYyBvmPlBnNaocduDm4lRdjJrl/01b8Jmje5HeYdbUTBkwyiFLk91PHJktATKY0= ceph-0518de98-763b-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T03:57:48.166 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu0BwHdO5ZVgfJ6EjY87o5xBngC73LF1kcfsekz1IrsZC4kGdwLn1WRVIPGzDR2znF3uBJDbj78Sncc7xVFKjIcnMqZYpwLXTE9DXA02s/il1El/enDUN6lwvtOQ1eYdO+wjaEbO5lVdZnjczeW5j7XWQ+v2TZo5YB6Tf8SS6hsuNHNASZJgpj/rP40+W2WtYdsuBaEhherzP1cdJrmU8mJhek5BAQo3h9PHQ1vm8PZq+Ca4EnnWMQJhpeafSfbPmAD1kqbuwhDPucyU820VyG7znzaTq3Bxlyf9PcwLvvZQGye9V3B/fxuy/cNZrHMDq7gRxIYkb8DPffmnpx3zkOI9Djqe4ynDWwLhuRj1UNBnzyMyhBi8Z/kY934l1rrzZUe4lP0NzSpXxunJrgvGHMb2lXL6sHr7vdHbSsvctdhwqtVCD+tae35FWhh7zmGft7MYyBvmPlBnNaocduDm4lRdjJrl/01b8Jmje5HeYdbUTBkwyiFLk91PHJktATKY0= ceph-0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:57:48.200 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu0BwHdO5ZVgfJ6EjY87o5xBngC73LF1kcfsekz1IrsZC4kGdwLn1WRVIPGzDR2znF3uBJDbj78Sncc7xVFKjIcnMqZYpwLXTE9DXA02s/il1El/enDUN6lwvtOQ1eYdO+wjaEbO5lVdZnjczeW5j7XWQ+v2TZo5YB6Tf8SS6hsuNHNASZJgpj/rP40+W2WtYdsuBaEhherzP1cdJrmU8mJhek5BAQo3h9PHQ1vm8PZq+Ca4EnnWMQJhpeafSfbPmAD1kqbuwhDPucyU820VyG7znzaTq3Bxlyf9PcwLvvZQGye9V3B/fxuy/cNZrHMDq7gRxIYkb8DPffmnpx3zkOI9Djqe4ynDWwLhuRj1UNBnzyMyhBi8Z/kY934l1rrzZUe4lP0NzSpXxunJrgvGHMb2lXL6sHr7vdHbSsvctdhwqtVCD+tae35FWhh7zmGft7MYyBvmPlBnNaocduDm4lRdjJrl/01b8Jmje5HeYdbUTBkwyiFLk91PHJktATKY0= ceph-0518de98-763b-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T03:57:48.242 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu0BwHdO5ZVgfJ6EjY87o5xBngC73LF1kcfsekz1IrsZC4kGdwLn1WRVIPGzDR2znF3uBJDbj78Sncc7xVFKjIcnMqZYpwLXTE9DXA02s/il1El/enDUN6lwvtOQ1eYdO+wjaEbO5lVdZnjczeW5j7XWQ+v2TZo5YB6Tf8SS6hsuNHNASZJgpj/rP40+W2WtYdsuBaEhherzP1cdJrmU8mJhek5BAQo3h9PHQ1vm8PZq+Ca4EnnWMQJhpeafSfbPmAD1kqbuwhDPucyU820VyG7znzaTq3Bxlyf9PcwLvvZQGye9V3B/fxuy/cNZrHMDq7gRxIYkb8DPffmnpx3zkOI9Djqe4ynDWwLhuRj1UNBnzyMyhBi8Z/kY934l1rrzZUe4lP0NzSpXxunJrgvGHMb2lXL6sHr7vdHbSsvctdhwqtVCD+tae35FWhh7zmGft7MYyBvmPlBnNaocduDm4lRdjJrl/01b8Jmje5HeYdbUTBkwyiFLk91PHJktATKY0= ceph-0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:57:48.255 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu0BwHdO5ZVgfJ6EjY87o5xBngC73LF1kcfsekz1IrsZC4kGdwLn1WRVIPGzDR2znF3uBJDbj78Sncc7xVFKjIcnMqZYpwLXTE9DXA02s/il1El/enDUN6lwvtOQ1eYdO+wjaEbO5lVdZnjczeW5j7XWQ+v2TZo5YB6Tf8SS6hsuNHNASZJgpj/rP40+W2WtYdsuBaEhherzP1cdJrmU8mJhek5BAQo3h9PHQ1vm8PZq+Ca4EnnWMQJhpeafSfbPmAD1kqbuwhDPucyU820VyG7znzaTq3Bxlyf9PcwLvvZQGye9V3B/fxuy/cNZrHMDq7gRxIYkb8DPffmnpx3zkOI9Djqe4ynDWwLhuRj1UNBnzyMyhBi8Z/kY934l1rrzZUe4lP0NzSpXxunJrgvGHMb2lXL6sHr7vdHbSsvctdhwqtVCD+tae35FWhh7zmGft7MYyBvmPlBnNaocduDm4lRdjJrl/01b8Jmje5HeYdbUTBkwyiFLk91PHJktATKY0= ceph-0518de98-763b-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T03:57:48.297 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu0BwHdO5ZVgfJ6EjY87o5xBngC73LF1kcfsekz1IrsZC4kGdwLn1WRVIPGzDR2znF3uBJDbj78Sncc7xVFKjIcnMqZYpwLXTE9DXA02s/il1El/enDUN6lwvtOQ1eYdO+wjaEbO5lVdZnjczeW5j7XWQ+v2TZo5YB6Tf8SS6hsuNHNASZJgpj/rP40+W2WtYdsuBaEhherzP1cdJrmU8mJhek5BAQo3h9PHQ1vm8PZq+Ca4EnnWMQJhpeafSfbPmAD1kqbuwhDPucyU820VyG7znzaTq3Bxlyf9PcwLvvZQGye9V3B/fxuy/cNZrHMDq7gRxIYkb8DPffmnpx3zkOI9Djqe4ynDWwLhuRj1UNBnzyMyhBi8Z/kY934l1rrzZUe4lP0NzSpXxunJrgvGHMb2lXL6sHr7vdHbSsvctdhwqtVCD+tae35FWhh7zmGft7MYyBvmPlBnNaocduDm4lRdjJrl/01b8Jmje5HeYdbUTBkwyiFLk91PHJktATKY0= ceph-0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:57:48.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T03:57:48.566 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:57:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.104+0000 7fd403a5c640 1 -- 172.21.15.19:0/1370783683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 msgr2=0x7fd3fc101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.104+0000 7fd403a5c640 1 --2- 172.21.15.19:0/1370783683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 0x7fd3fc101a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009880 tx=0x7fd3f802a5d0 comp rx=0 tx=0).stop 2024-09-19T03:57:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.104+0000 7fd403a5c640 1 -- 172.21.15.19:0/1370783683 shutdown_connections 2024-09-19T03:57:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.104+0000 7fd403a5c640 1 --2- 172.21.15.19:0/1370783683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 0x7fd3fc101a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.104+0000 7fd403a5c640 1 -- 172.21.15.19:0/1370783683 >> 172.21.15.19:0/1370783683 conn(0x7fd3fc0fcdb0 msgr2=0x7fd3fc0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.104+0000 7fd403a5c640 1 -- 172.21.15.19:0/1370783683 shutdown_connections 2024-09-19T03:57:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.104+0000 7fd403a5c640 1 -- 172.21.15.19:0/1370783683 wait complete. 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.105+0000 7fd403a5c640 1 Processor -- start 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.105+0000 7fd403a5c640 1 -- start start 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.105+0000 7fd403a5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 0x7fd3fc075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.105+0000 7fd403a5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc107730 con 0x7fd3fc101620 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.105+0000 7fd4017f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 0x7fd3fc075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.105+0000 7fd4017f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 0x7fd3fc075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43962/0 (socket says 172.21.15.19:43962) 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.105+0000 7fd4017f8640 1 -- 172.21.15.19:0/3500484480 learned_addr learned my addr 172.21.15.19:0/3500484480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.106+0000 7fd4017f8640 1 -- 172.21.15.19:0/3500484480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc0763f0 con 0x7fd3fc101620 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.106+0000 7fd4017f8640 1 --2- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 0x7fd3fc075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8004f60 tx=0x7fd3f80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.106+0000 7fd3f27fc640 1 -- 172.21.15.19:0/3500484480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f803f070 con 0x7fd3fc101620 2024-09-19T03:57:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.106+0000 7fd3f27fc640 1 -- 172.21.15.19:0/3500484480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f8004d40 con 0x7fd3fc101620 2024-09-19T03:57:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.106+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc0745e0 con 0x7fd3fc101620 2024-09-19T03:57:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.106+0000 7fd3f27fc640 1 -- 172.21.15.19:0/3500484480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f8035030 con 0x7fd3fc101620 2024-09-19T03:57:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.106+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc074b20 con 0x7fd3fc101620 2024-09-19T03:57:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.107+0000 7fd3f27fc640 1 -- 172.21.15.19:0/3500484480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd3f8042070 con 0x7fd3fc101620 2024-09-19T03:57:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.107+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3fc107eb0 con 0x7fd3fc101620 2024-09-19T03:57:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.107+0000 7fd3f27fc640 1 --2- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd3e8039300 0x7fd3e803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.108+0000 7fd3f27fc640 1 -- 172.21.15.19:0/3500484480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd3f806cfa0 con 0x7fd3fc101620 2024-09-19T03:57:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.108+0000 7fd400ff7640 1 --2- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd3e8039300 0x7fd3e803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.108+0000 7fd400ff7640 1 --2- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd3e8039300 0x7fd3e803b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0098c0 tx=0x7fd3ec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:49.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.114+0000 7fd3f27fc640 1 -- 172.21.15.19:0/3500484480 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd3f804a260 con 0x7fd3fc101620 2024-09-19T03:57:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.290+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fd3fc1901d0 con 0x7fd3fc101620 2024-09-19T03:57:49.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.292+0000 7fd3f27fc640 1 -- 172.21.15.19:0/3500484480 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fd3f8003e10 con 0x7fd3fc101620 2024-09-19T03:57:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd3e8039300 msgr2=0x7fd3e803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 --2- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd3e8039300 0x7fd3e803b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0098c0 tx=0x7fd3ec006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 msgr2=0x7fd3fc075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 --2- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 0x7fd3fc075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8004f60 tx=0x7fd3f80048f0 comp rx=0 tx=0).stop 2024-09-19T03:57:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 shutdown_connections 2024-09-19T03:57:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 --2- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd3e8039300 0x7fd3e803b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 --2- 172.21.15.19:0/3500484480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3fc101620 0x7fd3fc075eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 >> 172.21.15.19:0/3500484480 conn(0x7fd3fc0fcdb0 msgr2=0x7fd3fc0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 shutdown_connections 2024-09-19T03:57:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:49.295+0000 7fd403a5c640 1 -- 172.21.15.19:0/3500484480 wait complete. 2024-09-19T03:57:50.459 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T03:57:50.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T03:57:50.581 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:50 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3500484480' entity='client.admin' 2024-09-19T03:57:50.757 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:57:51.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.224+0000 7f2366a68640 1 -- 172.21.15.19:0/3647976744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 msgr2=0x7f2360100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:51.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.224+0000 7f2366a68640 1 --2- 172.21.15.19:0/3647976744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 0x7f2360100940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2354009880 tx=0x7f235402a5d0 comp rx=0 tx=0).stop 2024-09-19T03:57:51.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.224+0000 7f2366a68640 1 -- 172.21.15.19:0/3647976744 shutdown_connections 2024-09-19T03:57:51.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.224+0000 7f2366a68640 1 --2- 172.21.15.19:0/3647976744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 0x7f2360100940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:51.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.224+0000 7f2366a68640 1 -- 172.21.15.19:0/3647976744 >> 172.21.15.19:0/3647976744 conn(0x7f23600fbcd0 msgr2=0x7f23600fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:51.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.224+0000 7f2366a68640 1 -- 172.21.15.19:0/3647976744 shutdown_connections 2024-09-19T03:57:51.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.224+0000 7f2366a68640 1 -- 172.21.15.19:0/3647976744 wait complete. 2024-09-19T03:57:51.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.225+0000 7f2366a68640 1 Processor -- start 2024-09-19T03:57:51.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.225+0000 7f2366a68640 1 -- start start 2024-09-19T03:57:51.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.225+0000 7f2366a68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 0x7f2360198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:51.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.225+0000 7f2366a68640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23601065f0 con 0x7f2360100540 2024-09-19T03:57:51.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.225+0000 7f2364804640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 0x7f2360198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:51.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.225+0000 7f2364804640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 0x7f2360198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43984/0 (socket says 172.21.15.19:43984) 2024-09-19T03:57:51.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.225+0000 7f2364804640 1 -- 172.21.15.19:0/1451759974 learned_addr learned my addr 172.21.15.19:0/1451759974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:51.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.226+0000 7f2364804640 1 -- 172.21.15.19:0/1451759974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23601994f0 con 0x7f2360100540 2024-09-19T03:57:51.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.226+0000 7f2364804640 1 --2- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 0x7f2360198fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2354004f60 tx=0x7f23540048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:51.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.226+0000 7f235dffb640 1 -- 172.21.15.19:0/1451759974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235403f070 con 0x7f2360100540 2024-09-19T03:57:51.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.226+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2360199780 con 0x7f2360100540 2024-09-19T03:57:51.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.226+0000 7f235dffb640 1 -- 172.21.15.19:0/1451759974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2354004d40 con 0x7f2360100540 2024-09-19T03:57:51.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.226+0000 7f235dffb640 1 -- 172.21.15.19:0/1451759974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2354035030 con 0x7f2360100540 2024-09-19T03:57:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.226+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236019c470 con 0x7f2360100540 2024-09-19T03:57:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.227+0000 7f235dffb640 1 -- 172.21.15.19:0/1451759974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f2354042070 con 0x7f2360100540 2024-09-19T03:57:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.227+0000 7f235dffb640 1 --2- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f23500392a0 0x7f235003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.228+0000 7f235dffb640 1 -- 172.21.15.19:0/1451759974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f235406e010 con 0x7f2360100540 2024-09-19T03:57:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.228+0000 7f235ffff640 1 --2- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f23500392a0 0x7f235003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.228+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2360106d70 con 0x7f2360100540 2024-09-19T03:57:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.228+0000 7f235ffff640 1 --2- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f23500392a0 0x7f235003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f234c0098c0 tx=0x7f234c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.232+0000 7f235dffb640 1 -- 172.21.15.19:0/1451759974 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2354040260 con 0x7f2360100540 2024-09-19T03:57:51.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.412+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f23600fd340 con 0x7f23500392a0 2024-09-19T03:57:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.415+0000 7f235dffb640 1 -- 172.21.15.19:0/1451759974 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f23600fd340 con 0x7f23500392a0 2024-09-19T03:57:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f23500392a0 msgr2=0x7f235003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 --2- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f23500392a0 0x7f235003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f234c0098c0 tx=0x7f234c006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 msgr2=0x7f2360198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 --2- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 0x7f2360198fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2354004f60 tx=0x7f23540048f0 comp rx=0 tx=0).stop 2024-09-19T03:57:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 shutdown_connections 2024-09-19T03:57:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 --2- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f23500392a0 0x7f235003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 --2- 172.21.15.19:0/1451759974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2360100540 0x7f2360198fb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 >> 172.21.15.19:0/1451759974 conn(0x7f23600fbcd0 msgr2=0x7f2360069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 shutdown_connections 2024-09-19T03:57:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:51.419+0000 7f2366a68640 1 -- 172.21.15.19:0/1451759974 wait complete. 2024-09-19T03:57:52.011 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi111 2024-09-19T03:57:52.011 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:57:52.011 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.conf 2024-09-19T03:57:52.030 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:57:52.030 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:57:52.088 INFO:tasks.cephadm:Adding host smithi111 to orchestrator... 2024-09-19T03:57:52.088 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch host add smithi111 2024-09-19T03:57:52.478 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:57:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:52 smithi019 ceph-mon[30938]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:53.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.012+0000 7f243fdab640 1 -- 172.21.15.19:0/2990315862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 msgr2=0x7f24381044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:53.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.012+0000 7f243fdab640 1 --2- 172.21.15.19:0/2990315862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 0x7f24381044c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2434009880 tx=0x7f243402a5d0 comp rx=0 tx=0).stop 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.013+0000 7f243fdab640 1 -- 172.21.15.19:0/2990315862 shutdown_connections 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.013+0000 7f243fdab640 1 --2- 172.21.15.19:0/2990315862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 0x7f24381044c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.013+0000 7f243fdab640 1 -- 172.21.15.19:0/2990315862 >> 172.21.15.19:0/2990315862 conn(0x7f24380fbcd0 msgr2=0x7f24380fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.013+0000 7f243fdab640 1 -- 172.21.15.19:0/2990315862 shutdown_connections 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.013+0000 7f243fdab640 1 -- 172.21.15.19:0/2990315862 wait complete. 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.013+0000 7f243fdab640 1 Processor -- start 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.013+0000 7f243fdab640 1 -- start start 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.014+0000 7f243fdab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 0x7f2438101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.014+0000 7f243fdab640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2438107670 con 0x7f24381020d0 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.014+0000 7f243db47640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 0x7f2438101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:53.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.014+0000 7f243db47640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 0x7f2438101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60782/0 (socket says 172.21.15.19:60782) 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.014+0000 7f243db47640 1 -- 172.21.15.19:0/2323988039 learned_addr learned my addr 172.21.15.19:0/2323988039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.014+0000 7f243db47640 1 -- 172.21.15.19:0/2323988039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2438101e90 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.014+0000 7f243db47640 1 --2- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 0x7f2438101950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2434009850 tx=0x7f24340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.014+0000 7f242effd640 1 -- 172.21.15.19:0/2323988039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f243403f070 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.015+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2438100080 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.015+0000 7f242effd640 1 -- 172.21.15.19:0/2323988039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2434003ad0 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.015+0000 7f242effd640 1 -- 172.21.15.19:0/2323988039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2434035030 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.015+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24381005c0 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.016+0000 7f242effd640 1 -- 172.21.15.19:0/2323988039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f2434042070 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.016+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2438107df0 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.016+0000 7f242effd640 1 --2- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f24240392f0 0x7f242403b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.016+0000 7f242effd640 1 -- 172.21.15.19:0/2323988039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f243406d1f0 con 0x7f24381020d0 2024-09-19T03:57:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.016+0000 7f243d346640 1 --2- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f24240392f0 0x7f242403b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:53.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.016+0000 7f243d346640 1 --2- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f24240392f0 0x7f242403b7b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24280098c0 tx=0x7f2428006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:53.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.022+0000 7f242effd640 1 -- 172.21.15.19:0/2323988039 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f243402f300 con 0x7f24381020d0 2024-09-19T03:57:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:53.203+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}) -- 0x7f2438190350 con 0x7f24240392f0 2024-09-19T03:57:53.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:53.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: Updating smithi019:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: Updating smithi019:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:57:53.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:57:53.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:57:53.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:55.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:55 smithi019 ceph-mon[30938]: Deploying cephadm binary to smithi111 2024-09-19T03:57:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.085+0000 7f242effd640 1 -- 172.21.15.19:0/2323988039 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2438190350 con 0x7f24240392f0 2024-09-19T03:57:57.087 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi111' with addr '172.21.15.111' 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f24240392f0 msgr2=0x7f242403b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 --2- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f24240392f0 0x7f242403b7b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24280098c0 tx=0x7f2428006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 msgr2=0x7f2438101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 --2- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 0x7f2438101950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2434009850 tx=0x7f24340048f0 comp rx=0 tx=0).stop 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 shutdown_connections 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 --2- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f24240392f0 0x7f242403b7b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 --2- 172.21.15.19:0/2323988039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24381020d0 0x7f2438101950 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 >> 172.21.15.19:0/2323988039 conn(0x7f24380fbcd0 msgr2=0x7f24380fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.087+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 shutdown_connections 2024-09-19T03:57:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:57.088+0000 7f243fdab640 1 -- 172.21.15.19:0/2323988039 wait complete. 2024-09-19T03:57:57.693 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T03:57:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:57:58.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:58 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:58.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:58 smithi019 ceph-mon[30938]: Added host smithi111 2024-09-19T03:57:58.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:58 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:57:58.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:58 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:58.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:58 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:58.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.434+0000 7fb284c10640 1 -- 172.21.15.19:0/1875207845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 msgr2=0x7fb280102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:58.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.434+0000 7fb284c10640 1 --2- 172.21.15.19:0/1875207845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 0x7fb280102b70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb270009810 tx=0x7fb27002a540 comp rx=0 tx=0).stop 2024-09-19T03:57:58.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.434+0000 7fb284c10640 1 -- 172.21.15.19:0/1875207845 shutdown_connections 2024-09-19T03:57:58.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.434+0000 7fb284c10640 1 --2- 172.21.15.19:0/1875207845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 0x7fb280102b70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.434+0000 7fb284c10640 1 -- 172.21.15.19:0/1875207845 >> 172.21.15.19:0/1875207845 conn(0x7fb280077310 msgr2=0x7fb2800fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.434+0000 7fb284c10640 1 -- 172.21.15.19:0/1875207845 shutdown_connections 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.434+0000 7fb284c10640 1 -- 172.21.15.19:0/1875207845 wait complete. 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.435+0000 7fb284c10640 1 Processor -- start 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.435+0000 7fb284c10640 1 -- start start 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.435+0000 7fb284c10640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 0x7fb280197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.435+0000 7fb284c10640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb28010bc80 con 0x7fb280102790 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.435+0000 7fb27e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 0x7fb280197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.435+0000 7fb27e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 0x7fb280197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60798/0 (socket says 172.21.15.19:60798) 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.435+0000 7fb27e59c640 1 -- 172.21.15.19:0/2990732378 learned_addr learned my addr 172.21.15.19:0/2990732378 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:57:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.435+0000 7fb27e59c640 1 -- 172.21.15.19:0/2990732378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2801976d0 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.436+0000 7fb27e59c640 1 --2- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 0x7fb280197190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2700045e0 tx=0x7fb270004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.436+0000 7fb26f7fe640 1 -- 172.21.15.19:0/2990732378 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb270030e70 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.436+0000 7fb26f7fe640 1 -- 172.21.15.19:0/2990732378 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2700318e0 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.436+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb280197960 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.436+0000 7fb26f7fe640 1 -- 172.21.15.19:0/2990732378 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2700049c0 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.436+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb280197dc0 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.437+0000 7fb26f7fe640 1 -- 172.21.15.19:0/2990732378 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb270039040 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.437+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb280069f30 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.437+0000 7fb26f7fe640 1 --2- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb268039300 0x7fb26803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.437+0000 7fb26f7fe640 1 -- 172.21.15.19:0/2990732378 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb27006d580 con 0x7fb280102790 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.437+0000 7fb27dd9b640 1 --2- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb268039300 0x7fb26803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.437+0000 7fb27dd9b640 1 --2- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb268039300 0x7fb26803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb2740098c0 tx=0x7fb274006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:57:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.442+0000 7fb26f7fe640 1 -- 172.21.15.19:0/2990732378 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb270040030 con 0x7fb280102790 2024-09-19T03:57:58.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.634+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2801984f0 con 0x7fb268039300 2024-09-19T03:57:58.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.635+0000 7fb26f7fe640 1 -- 172.21.15.19:0/2990732378 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fb2801984f0 con 0x7fb268039300 2024-09-19T03:57:58.637 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:57:58.637 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}] 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.637+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb268039300 msgr2=0x7fb26803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.637+0000 7fb284c10640 1 --2- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb268039300 0x7fb26803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb2740098c0 tx=0x7fb274006db0 comp rx=0 tx=0).stop 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.637+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 msgr2=0x7fb280197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.637+0000 7fb284c10640 1 --2- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 0x7fb280197190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2700045e0 tx=0x7fb270004610 comp rx=0 tx=0).stop 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.637+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 shutdown_connections 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.637+0000 7fb284c10640 1 --2- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb268039300 0x7fb26803b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.637+0000 7fb284c10640 1 --2- 172.21.15.19:0/2990732378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb280102790 0x7fb280197190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.637+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 >> 172.21.15.19:0/2990732378 conn(0x7fb280077310 msgr2=0x7fb280105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:57:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.638+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 shutdown_connections 2024-09-19T03:57:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:58.638+0000 7fb284c10640 1 -- 172.21.15.19:0/2990732378 wait complete. 2024-09-19T03:57:59.245 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-09-19T03:57:59.245 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:57:59.245 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-09-19T03:57:59.264 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:57:59.264 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:57:59.321 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-09-19T03:57:59.322 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch host add smithi196 2024-09-19T03:57:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:57:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:57:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:57:59 smithi019 ceph-mon[30938]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T03:58:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.997+0000 7fc9dcd38640 1 -- 172.21.15.19:0/2197530883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 msgr2=0x7fc9d8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.997+0000 7fc9dcd38640 1 --2- 172.21.15.19:0/2197530883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 0x7fc9d8106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009880 tx=0x7fc9c802a5d0 comp rx=0 tx=0).stop 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.997+0000 7fc9dcd38640 1 -- 172.21.15.19:0/2197530883 shutdown_connections 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.997+0000 7fc9dcd38640 1 --2- 172.21.15.19:0/2197530883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 0x7fc9d8106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.997+0000 7fc9dcd38640 1 -- 172.21.15.19:0/2197530883 >> 172.21.15.19:0/2197530883 conn(0x7fc9d80fc1b0 msgr2=0x7fc9d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.997+0000 7fc9dcd38640 1 -- 172.21.15.19:0/2197530883 shutdown_connections 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.997+0000 7fc9dcd38640 1 -- 172.21.15.19:0/2197530883 wait complete. 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.998+0000 7fc9dcd38640 1 Processor -- start 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.998+0000 7fc9dcd38640 1 -- start start 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9dcd38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 0x7fc9d8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9dcd38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d810ab90 con 0x7fc9d8106560 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9d659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 0x7fc9d8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9d659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 0x7fc9d8197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60810/0 (socket says 172.21.15.19:60810) 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9d659c640 1 -- 172.21.15.19:0/3150855530 learned_addr learned my addr 172.21.15.19:0/3150855530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:58:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9d659c640 1 -- 172.21.15.19:0/3150855530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d8198490 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9d659c640 1 --2- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 0x7fc9d8197f50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009230 tx=0x7fc9c8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9c77fe640 1 -- 172.21.15.19:0/3150855530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9c803f070 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d8198720 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:57:59.999+0000 7fc9c77fe640 1 -- 172.21.15.19:0/3150855530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c803bc80 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.000+0000 7fc9c77fe640 1 -- 172.21.15.19:0/3150855530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9c8035030 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.000+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d8198b80 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.000+0000 7fc9c77fe640 1 -- 172.21.15.19:0/3150855530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc9c8042070 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.001+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d8101bd0 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.001+0000 7fc9c77fe640 1 --2- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc9c00392a0 0x7fc9c003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.001+0000 7fc9c77fe640 1 -- 172.21.15.19:0/3150855530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc9c806ce40 con 0x7fc9d8106560 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.001+0000 7fc9d5d9b640 1 --2- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc9c00392a0 0x7fc9c003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.001+0000 7fc9d5d9b640 1 --2- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc9c00392a0 0x7fc9c003b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098c0 tx=0x7fc9cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.005+0000 7fc9c77fe640 1 -- 172.21.15.19:0/3150855530 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc9c80314c0 con 0x7fc9d8106560 2024-09-19T03:58:00.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:00.186+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7fc9d81069c0 con 0x7fc9c00392a0 2024-09-19T03:58:00.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:00 smithi019 ceph-mon[30938]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:01.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:01 smithi019 ceph-mon[30938]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:01.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:58:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:03.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:03 smithi019 ceph-mon[30938]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-19T03:58:03.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:03 smithi019 ceph-mon[30938]: Deploying cephadm binary to smithi196 2024-09-19T03:58:03.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:03 smithi019 ceph-mon[30938]: Updating smithi111:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:03.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:03 smithi019 ceph-mon[30938]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:58:03.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:03 smithi019 ceph-mon[30938]: Updating smithi111:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T03:58:03.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:03.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:03.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:04.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.050+0000 7fc9c77fe640 1 -- 172.21.15.19:0/3150855530 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc9d81069c0 con 0x7fc9c00392a0 2024-09-19T03:58:04.053 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.052+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc9c00392a0 msgr2=0x7fc9c003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.052+0000 7fc9dcd38640 1 --2- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc9c00392a0 0x7fc9c003b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098c0 tx=0x7fc9cc006db0 comp rx=0 tx=0).stop 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.052+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 msgr2=0x7fc9d8197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.052+0000 7fc9dcd38640 1 --2- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 0x7fc9d8197f50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009230 tx=0x7fc9c8003d40 comp rx=0 tx=0).stop 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.053+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 shutdown_connections 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.053+0000 7fc9dcd38640 1 --2- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc9c00392a0 0x7fc9c003b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.053+0000 7fc9dcd38640 1 --2- 172.21.15.19:0/3150855530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9d8106560 0x7fc9d8197f50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.053+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 >> 172.21.15.19:0/3150855530 conn(0x7fc9d80fc1b0 msgr2=0x7fc9d80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.053+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 shutdown_connections 2024-09-19T03:58:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:04.053+0000 7fc9dcd38640 1 -- 172.21.15.19:0/3150855530 wait complete. 2024-09-19T03:58:04.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:04 smithi019 ceph-mon[30938]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:04.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:04 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:04.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:04 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:04.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T03:58:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:58:05.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:05 smithi019 ceph-mon[30938]: Added host smithi196 2024-09-19T03:58:05.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:05.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.371+0000 7fdb39d32640 1 -- 172.21.15.19:0/1800030212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 msgr2=0x7fdb34102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.371+0000 7fdb39d32640 1 --2- 172.21.15.19:0/1800030212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 0x7fdb34102b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009810 tx=0x7fdb2402a540 comp rx=0 tx=0).stop 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.371+0000 7fdb39d32640 1 -- 172.21.15.19:0/1800030212 shutdown_connections 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.371+0000 7fdb39d32640 1 --2- 172.21.15.19:0/1800030212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 0x7fdb34102b10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.371+0000 7fdb39d32640 1 -- 172.21.15.19:0/1800030212 >> 172.21.15.19:0/1800030212 conn(0x7fdb34077310 msgr2=0x7fdb340fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.371+0000 7fdb39d32640 1 -- 172.21.15.19:0/1800030212 shutdown_connections 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.371+0000 7fdb39d32640 1 -- 172.21.15.19:0/1800030212 wait complete. 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.372+0000 7fdb39d32640 1 Processor -- start 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.372+0000 7fdb39d32640 1 -- start start 2024-09-19T03:58:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.372+0000 7fdb39d32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 0x7fdb341971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.372+0000 7fdb39d32640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3410bc80 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.372+0000 7fdb337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 0x7fdb341971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.372+0000 7fdb337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 0x7fdb341971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45582/0 (socket says 172.21.15.19:45582) 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.372+0000 7fdb337fe640 1 -- 172.21.15.19:0/3743065609 learned_addr learned my addr 172.21.15.19:0/3743065609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.372+0000 7fdb337fe640 1 -- 172.21.15.19:0/3743065609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb341976f0 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.373+0000 7fdb337fe640 1 --2- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 0x7fdb341971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb240045e0 tx=0x7fdb24004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.373+0000 7fdb30ff9640 1 -- 172.21.15.19:0/3743065609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb24030e70 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.373+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb34197980 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.373+0000 7fdb30ff9640 1 -- 172.21.15.19:0/3743065609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb240318e0 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.373+0000 7fdb30ff9640 1 -- 172.21.15.19:0/3743065609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb240049c0 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.373+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb34197de0 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.374+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb34069f90 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.374+0000 7fdb30ff9640 1 -- 172.21.15.19:0/3743065609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fdb24039040 con 0x7fdb34102730 2024-09-19T03:58:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.374+0000 7fdb30ff9640 1 --2- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdb1c0392a0 0x7fdb1c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:05.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.374+0000 7fdb30ff9640 1 -- 172.21.15.19:0/3743065609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb2406d590 con 0x7fdb34102730 2024-09-19T03:58:05.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.374+0000 7fdb32ffd640 1 --2- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdb1c0392a0 0x7fdb1c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:05.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.375+0000 7fdb32ffd640 1 --2- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdb1c0392a0 0x7fdb1c03b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb280098c0 tx=0x7fdb28006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:05.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.379+0000 7fdb30ff9640 1 -- 172.21.15.19:0/3743065609 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdb24040030 con 0x7fdb34102730 2024-09-19T03:58:05.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.562+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb34198570 con 0x7fdb1c0392a0 2024-09-19T03:58:05.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.564+0000 7fdb30ff9640 1 -- 172.21.15.19:0/3743065609 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fdb34198570 con 0x7fdb1c0392a0 2024-09-19T03:58:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:58:05.566 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-09-19T03:58:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdb1c0392a0 msgr2=0x7fdb1c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 --2- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdb1c0392a0 0x7fdb1c03b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb280098c0 tx=0x7fdb28006db0 comp rx=0 tx=0).stop 2024-09-19T03:58:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 msgr2=0x7fdb341971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 --2- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 0x7fdb341971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb240045e0 tx=0x7fdb24004610 comp rx=0 tx=0).stop 2024-09-19T03:58:05.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 shutdown_connections 2024-09-19T03:58:05.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 --2- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdb1c0392a0 0x7fdb1c03b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:05.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 --2- 172.21.15.19:0/3743065609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb34102730 0x7fdb341971b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:05.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 >> 172.21.15.19:0/3743065609 conn(0x7fdb34077310 msgr2=0x7fdb341034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:05.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 shutdown_connections 2024-09-19T03:58:05.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:05.566+0000 7fdb39d32640 1 -- 172.21.15.19:0/3743065609 wait complete. 2024-09-19T03:58:06.163 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T03:58:06.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T03:58:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:58:06.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:06 smithi019 ceph-mon[30938]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:06.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:06 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:06.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.819+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/3471546317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 msgr2=0x7fe8e8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:06.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.819+0000 7fe8ee7e1640 1 --2- 172.21.15.19:0/3471546317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 0x7fe8e8107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009880 tx=0x7fe8d802a5d0 comp rx=0 tx=0).stop 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.819+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/3471546317 shutdown_connections 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.819+0000 7fe8ee7e1640 1 --2- 172.21.15.19:0/3471546317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 0x7fe8e8107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.819+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/3471546317 >> 172.21.15.19:0/3471546317 conn(0x7fe8e80fd290 msgr2=0x7fe8e80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.819+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/3471546317 shutdown_connections 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.819+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/3471546317 wait complete. 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.820+0000 7fe8ee7e1640 1 Processor -- start 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.820+0000 7fe8ee7e1640 1 -- start start 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.820+0000 7fe8ee7e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 0x7fe8e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.820+0000 7fe8ee7e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e810bc90 con 0x7fe8e8107660 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.820+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 0x7fe8e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.820+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 0x7fe8e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45594/0 (socket says 172.21.15.19:45594) 2024-09-19T03:58:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.820+0000 7fe8e7fff640 1 -- 172.21.15.19:0/1703580415 learned_addr learned my addr 172.21.15.19:0/1703580415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.821+0000 7fe8e7fff640 1 -- 172.21.15.19:0/1703580415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e80763e0 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.821+0000 7fe8e7fff640 1 --2- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 0x7fe8e80744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009230 tx=0x7fe8d8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.821+0000 7fe8e57fa640 1 -- 172.21.15.19:0/1703580415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d803f070 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.821+0000 7fe8e57fa640 1 -- 172.21.15.19:0/1703580415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d803bd40 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.821+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e8074a30 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.821+0000 7fe8e57fa640 1 -- 172.21.15.19:0/1703580415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d8035030 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.821+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e8074ef0 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.822+0000 7fe8e57fa640 1 -- 172.21.15.19:0/1703580415 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fe8d8042070 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.822+0000 7fe8e57fa640 1 --2- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8d00392a0 0x7fe8d003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.822+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e8102cd0 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.822+0000 7fe8e57fa640 1 -- 172.21.15.19:0/1703580415 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe8d806d180 con 0x7fe8e8107660 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.822+0000 7fe8e77fe640 1 --2- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8d00392a0 0x7fe8d003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.823+0000 7fe8e77fe640 1 --2- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8d00392a0 0x7fe8d003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0098c0 tx=0x7fe8dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:06.827+0000 7fe8e57fa640 1 -- 172.21.15.19:0/1703580415 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8d803a890 con 0x7fe8e8107660 2024-09-19T03:58:07.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.007+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe8e8075a20 con 0x7fe8e8107660 2024-09-19T03:58:07.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.181+0000 7fe8e57fa640 1 -- 172.21.15.19:0/1703580415 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fe8d803a3c0 con 0x7fe8e8107660 2024-09-19T03:58:07.184 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-09-19T03:58:07.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.183+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8d00392a0 msgr2=0x7fe8d003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:07.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.183+0000 7fe8ee7e1640 1 --2- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8d00392a0 0x7fe8d003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0098c0 tx=0x7fe8dc006db0 comp rx=0 tx=0).stop 2024-09-19T03:58:07.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.184+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 msgr2=0x7fe8e80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:07.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.184+0000 7fe8ee7e1640 1 --2- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 0x7fe8e80744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009230 tx=0x7fe8d8003d80 comp rx=0 tx=0).stop 2024-09-19T03:58:07.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.184+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 shutdown_connections 2024-09-19T03:58:07.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.184+0000 7fe8ee7e1640 1 --2- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8d00392a0 0x7fe8d003b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:07.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.184+0000 7fe8ee7e1640 1 --2- 172.21.15.19:0/1703580415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8107660 0x7fe8e80744f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:07.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.184+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 >> 172.21.15.19:0/1703580415 conn(0x7fe8e80fd290 msgr2=0x7fe8e80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:07.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.184+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 shutdown_connections 2024-09-19T03:58:07.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:07.184+0000 7fe8ee7e1640 1 -- 172.21.15.19:0/1703580415 wait complete. 2024-09-19T03:58:07.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:07 smithi019 ceph-mon[30938]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T03:58:07.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:07 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1703580415' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T03:58:07.795 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-09-19T03:58:07.795 INFO:tasks.cephadm:Adding mon.b on smithi111 2024-09-19T03:58:07.796 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-09-19T03:58:07.796 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi111:172.21.15.111=b;smithi196:172.21.15.196=c' 2024-09-19T03:58:08.481 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T03:58:08.524 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T03:58:08.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:08 smithi019 ceph-mon[30938]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:08.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:08 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1703580415' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T03:58:08.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:08 smithi019 ceph-mon[30938]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T03:58:09.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 -- 172.21.15.196:0/238845267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 msgr2=0x7f1560101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:09.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 --2- 172.21.15.196:0/238845267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 0x7f1560101f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1550009880 tx=0x7f155002a5d0 comp rx=0 tx=0).stop 2024-09-19T03:58:09.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 -- 172.21.15.196:0/238845267 shutdown_connections 2024-09-19T03:58:09.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 --2- 172.21.15.196:0/238845267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 0x7f1560101f20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:09.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 -- 172.21.15.196:0/238845267 >> 172.21.15.196:0/238845267 conn(0x7f15600fb400 msgr2=0x7f15600fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:09.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 -- 172.21.15.196:0/238845267 shutdown_connections 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 -- 172.21.15.196:0/238845267 wait complete. 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 Processor -- start 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 -- start start 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 0x7f1560198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.991+0000 7f156651d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15601098c0 con 0x7f1560101b20 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.992+0000 7f155ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 0x7f1560198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.992+0000 7f155ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 0x7f1560198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.196:55264/0 (socket says 172.21.15.196:55264) 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.992+0000 7f155ffff640 1 -- 172.21.15.196:0/318401239 learned_addr learned my addr 172.21.15.196:0/318401239 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.992+0000 7f155ffff640 1 -- 172.21.15.196:0/318401239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1560199470 con 0x7f1560101b20 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.992+0000 7f155ffff640 1 --2- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 0x7f1560198f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1550004f60 tx=0x7f15500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:09.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.992+0000 7f155d7fa640 1 -- 172.21.15.196:0/318401239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f155003f070 con 0x7f1560101b20 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.993+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1560199760 con 0x7f1560101b20 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.993+0000 7f155d7fa640 1 -- 172.21.15.196:0/318401239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1550004d40 con 0x7f1560101b20 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.993+0000 7f155d7fa640 1 -- 172.21.15.196:0/318401239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1550035030 con 0x7f1560101b20 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.993+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156019c3f0 con 0x7f1560101b20 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.993+0000 7f155d7fa640 1 -- 172.21.15.196:0/318401239 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f1550042070 con 0x7f1560101b20 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.993+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f156010a040 con 0x7f1560101b20 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.994+0000 7f155d7fa640 1 --2- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15480392a0 0x7f154803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.994+0000 7f155d7fa640 1 -- 172.21.15.196:0/318401239 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f155006d1d0 con 0x7f1560101b20 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.994+0000 7f155f7fe640 1 --2- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15480392a0 0x7f154803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.994+0000 7f155f7fe640 1 --2- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15480392a0 0x7f154803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f15540098c0 tx=0x7f1554006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:09.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:08.999+0000 7f155d7fa640 1 -- 172.21.15.196:0/318401239 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1550042360 con 0x7f1560101b20 2024-09-19T03:58:09.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.186+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi111:172.21.15.111=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7f156002da70 con 0x7f15480392a0 2024-09-19T03:58:09.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.191+0000 7f155d7fa640 1 -- 172.21.15.196:0/318401239 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f156002da70 con 0x7f15480392a0 2024-09-19T03:58:09.193 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-09-19T03:58:09.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15480392a0 msgr2=0x7f154803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:09.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 --2- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15480392a0 0x7f154803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f15540098c0 tx=0x7f1554006db0 comp rx=0 tx=0).stop 2024-09-19T03:58:09.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 msgr2=0x7f1560198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:09.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 --2- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 0x7f1560198f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1550004f60 tx=0x7f15500048f0 comp rx=0 tx=0).stop 2024-09-19T03:58:09.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 shutdown_connections 2024-09-19T03:58:09.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 --2- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15480392a0 0x7f154803b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:09.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 --2- 172.21.15.196:0/318401239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1560101b20 0x7f1560198f30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:09.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 >> 172.21.15.196:0/318401239 conn(0x7f15600fb400 msgr2=0x7f15600fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:09.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 shutdown_connections 2024-09-19T03:58:09.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:09.193+0000 7f156651d640 1 -- 172.21.15.196:0/318401239 wait complete. 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:09.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-19T03:58:09.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: Updating smithi196:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:09.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:58:09.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: Updating smithi196:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T03:58:09.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi111:172.21.15.111=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:09.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi111:172.21.15.111=b;smithi196:172.21.15.196=c;count:3 2024-09-19T03:58:09.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:09.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:09.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:09.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:09.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:09.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:09.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:09 smithi019 ceph-mon[30938]: Deploying daemon mon.c on smithi196 2024-09-19T03:58:09.750 DEBUG:teuthology.orchestra.run.smithi111:mon.b> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T03:58:09.753 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T03:58:09.755 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T03:58:09.755 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T03:58:11.383 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:11.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:11 smithi019 ceph-mon[30938]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.769+0000 7f702ce6c640 1 -- 172.21.15.196:0/3285008595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 msgr2=0x7f7028106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.769+0000 7f702ce6c640 1 --2- 172.21.15.196:0/3285008595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 0x7f7028106940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7018009c20 tx=0x7f701802aa90 comp rx=0 tx=0).stop 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.769+0000 7f702ce6c640 1 -- 172.21.15.196:0/3285008595 shutdown_connections 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.769+0000 7f702ce6c640 1 --2- 172.21.15.196:0/3285008595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 0x7f7028106940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.769+0000 7f702ce6c640 1 -- 172.21.15.196:0/3285008595 >> 172.21.15.196:0/3285008595 conn(0x7f70280fc1b0 msgr2=0x7f70280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.769+0000 7f702ce6c640 1 -- 172.21.15.196:0/3285008595 shutdown_connections 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.769+0000 7f702ce6c640 1 -- 172.21.15.196:0/3285008595 wait complete. 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.770+0000 7f702ce6c640 1 Processor -- start 2024-09-19T03:58:11.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.770+0000 7f702ce6c640 1 -- start start 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.770+0000 7f702ce6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 0x7f7028197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.770+0000 7f702ce6c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f702810ab90 con 0x7f7028106560 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f702659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 0x7f7028197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f702659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 0x7f7028197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.196:52262/0 (socket says 172.21.15.196:52262) 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f702659c640 1 -- 172.21.15.196:0/1179379046 learned_addr learned my addr 172.21.15.196:0/1179379046 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f702659c640 1 -- 172.21.15.196:0/1179379046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7028198500 con 0x7f7028106560 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f702659c640 1 --2- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 0x7f7028197fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7018005e30 tx=0x7f701802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f701f7fe640 1 -- 172.21.15.196:0/1179379046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7018047070 con 0x7f7028106560 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f701f7fe640 1 -- 172.21.15.196:0/1179379046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7018003df0 con 0x7f7028106560 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f701f7fe640 1 -- 172.21.15.196:0/1179379046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7018033050 con 0x7f7028106560 2024-09-19T03:58:11.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.771+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7028198790 con 0x7f7028106560 2024-09-19T03:58:11.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.772+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7028198bf0 con 0x7f7028106560 2024-09-19T03:58:11.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.773+0000 7f701f7fe640 1 -- 172.21.15.196:0/1179379046 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f701804a070 con 0x7f7028106560 2024-09-19T03:58:11.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.773+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7028101bd0 con 0x7f7028106560 2024-09-19T03:58:11.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.773+0000 7f701f7fe640 1 --2- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7014039300 0x7f701403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:11.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.773+0000 7f701f7fe640 1 -- 172.21.15.196:0/1179379046 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f701803e030 con 0x7f7028106560 2024-09-19T03:58:11.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.773+0000 7f7025d9b640 1 --2- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7014039300 0x7f701403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:11.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.774+0000 7f7025d9b640 1 --2- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7014039300 0x7f701403b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f70100098c0 tx=0x7f7010006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:11.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:11.778+0000 7f701f7fe640 1 -- 172.21.15.196:0/1179379046 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f701804a360 con 0x7f7028106560 2024-09-19T03:58:12.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.012+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f70281069c0 con 0x7f7028106560 2024-09-19T03:58:12.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.013+0000 7f701f7fe640 1 -- 172.21.15.196:0/1179379046 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f7018051300 con 0x7f7028106560 2024-09-19T03:58:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:58:12.015 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","modified":"2024-09-19T03:56:39.538905Z","created":"2024-09-19T03:56:39.538905Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T03:58:12.015 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7014039300 msgr2=0x7f701403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 --2- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7014039300 0x7f701403b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f70100098c0 tx=0x7f7010006db0 comp rx=0 tx=0).stop 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 msgr2=0x7f7028197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 --2- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 0x7f7028197fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7018005e30 tx=0x7f701802fc10 comp rx=0 tx=0).stop 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 shutdown_connections 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 --2- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7014039300 0x7f701403b7c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 --2- 172.21.15.196:0/1179379046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7028106560 0x7f7028197fc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 >> 172.21.15.196:0/1179379046 conn(0x7f70280fc1b0 msgr2=0x7f70280fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.015+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 shutdown_connections 2024-09-19T03:58:12.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:12.016+0000 7f702ce6c640 1 -- 172.21.15.196:0/1179379046 wait complete. 2024-09-19T03:58:12.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:12 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/1179379046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T03:58:13.616 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T03:58:13.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T03:58:13.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:13 smithi019 ceph-mon[30938]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:14.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 podman[35686]: 2024-09-19 03:58:14.197684586 +0000 UTC m=+0.347937028 container start 9a1e0c7faa3bdec14ff6b5e410ab1dae5df2df3aa19fba2f16ab15cc38ef70e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T03:58:14.311 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T03:58:14.548 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T03:58:14.548 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: pidfile_write: ignore empty --pid-file 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: load: jerasure load: lrc 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: RocksDB version: 7.9.2 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Git sha 0 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: DB SUMMARY 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: DB Session ID: 5WNKIN51KFSCIL02AC43 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: CURRENT file: CURRENT 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T03:58:14.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.error_if_exists: 0 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.create_if_missing: 0 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.paranoid_checks: 1 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.env: 0x563947b89940 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.info_log: 0x563948d576e0 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.statistics: (nil) 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.use_fsync: 0 2024-09-19T03:58:14.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_log_file_size: 0 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.allow_fallocate: 1 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.use_direct_reads: 0 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.db_log_dir: 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.wal_dir: 2024-09-19T03:58:14.551 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.write_buffer_manager: 0x563948d66dc0 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T03:58:14.553 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.unordered_write: 0 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.row_cache: None 2024-09-19T03:58:14.554 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.wal_filter: None 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.two_write_queues: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.wal_compression: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.atomic_flush: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.log_readahead_size: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T03:58:14.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_background_jobs: 2 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_background_compactions: -1 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_subcompactions: 1 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T03:58:14.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_open_files: -1 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_background_flushes: -1 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Compression algorithms supported: 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: kZSTD supported: 0 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: kXpressCompression supported: 0 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: kBZip2Compression supported: 0 2024-09-19T03:58:14.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: kLZ4Compression supported: 1 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: kZlibCompression supported: 1 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: kSnappyCompression supported: 1 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.merge_operator: 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_filter: None 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T03:58:14.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563948d57320) 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_type: 0 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_index_type: 0 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_shortening: 1 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: checksum: 4 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: no_block_cache: 0 2024-09-19T03:58:14.559 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache: 0x563948d4b1f0 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_options: 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: capacity : 536870912 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_shard_bits : 4 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: strict_capacity_limit : 0 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_compressed: (nil) 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: persistent_cache: (nil) 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size: 4096 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size_deviation: 10 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_restart_interval: 16 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_block_restart_interval: 1 2024-09-19T03:58:14.560 INFO:journalctl@ceph.mon.c.smithi196.stdout: metadata_block_size: 4096 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: partition_filters: 0 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: use_delta_encoding: 1 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: filter_policy: bloomfilter 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: whole_key_filtering: 1 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: verify_compression: 0 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: format_version: 5 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_index_compression: 1 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_align: 0 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: max_auto_readahead_size: 262144 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: prepopulate_block_cache: 0 2024-09-19T03:58:14.561 INFO:journalctl@ceph.mon.c.smithi196.stdout: initial_auto_readahead_size: 8192 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression: NoCompression 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.num_levels: 7 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T03:58:14.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T03:58:14.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T03:58:14.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T03:58:14.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.inplace_update_support: 0 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T03:58:14.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.bloom_locality: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.max_successive_merges: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.ttl: 2592000 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.enable_blob_files: false 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.min_blob_size: 0 2024-09-19T03:58:14.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 275e5354-9659-43d2-8493-50021c2b8f93 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718294224594, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T03:58:14.568 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718294225008, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726718294, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "275e5354-9659-43d2-8493-50021c2b8f93", "db_session_id": "5WNKIN51KFSCIL02AC43", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726718294225100, "job": 1, "event": "recovery_finished"} 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563948d78e00 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: DB pointer 0x563948e74000 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** DB Stats ** 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T03:58:14.569 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T03:58:14.570 INFO:journalctl@ceph.mon.c.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache BinnedLRUCache@0x563948d4b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T03:58:14.571 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: starting mon.c rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(???) e0 preinit fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T03:58:14.572 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout: btime 2024-09-19T03:56:42:596148+0000 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T03:58:14.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1703580415' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1703580415' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: Updating smithi196:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: Updating smithi196:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi111:172.21.15.111=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi111:172.21.15.111=b;smithi196:172.21.15.196=c;count:3 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:14.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:14.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:14.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: Deploying daemon mon.c on smithi196 2024-09-19T03:58:14.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:14.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/1179379046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T03:58:14.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:14.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 ceph-mon[35731]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T03:58:14.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 bash[35686]: 9a1e0c7faa3bdec14ff6b5e410ab1dae5df2df3aa19fba2f16ab15cc38ef70e4 2024-09-19T03:58:14.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:14 smithi196 systemd[1]: Started Ceph mon.c for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T03:58:19.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T03:58:19.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: mon.a calling monitor election 2024-09-19T03:58:19.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:19.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: mon.c calling monitor election 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: monmap epoch 2 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: last_changed 2024-09-19T03:58:14.249876+0000 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: created 2024-09-19T03:56:39.538905+0000 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: min_mon_release 19 (squid) 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: election_strategy: 1 2024-09-19T03:58:19.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: fsmap 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: mgrmap e13: x(active, since 38s) 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: overall HEALTH_OK 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:19.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:19 smithi019 ceph-mon[30938]: Deploying daemon mon.b on smithi111 2024-09-19T03:58:19.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: mon.a calling monitor election 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: mon.c calling monitor election 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: monmap epoch 2 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: last_changed 2024-09-19T03:58:14.249876+0000 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: created 2024-09-19T03:56:39.538905+0000 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: min_mon_release 19 (squid) 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: election_strategy: 1 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: fsmap 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: mgrmap e13: x(active, since 38s) 2024-09-19T03:58:19.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: overall HEALTH_OK 2024-09-19T03:58:19.711 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:19.711 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:19.711 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:19.711 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:19.711 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:19.711 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:19 smithi196 ceph-mon[35731]: Deploying daemon mon.b on smithi111 2024-09-19T03:58:20.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:20 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:20.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:20 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:20.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.747+0000 7f0dde37b640 1 -- 172.21.15.196:0/1514825374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 msgr2=0x7f0dc0008610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:20.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.747+0000 7f0dde37b640 1 --2- 172.21.15.196:0/1514825374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 0x7f0dc0008610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8069f20 tx=0x7f0dc802c4a0 comp rx=0 tx=0).stop 2024-09-19T03:58:20.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.747+0000 7f0dde37b640 1 -- 172.21.15.196:0/1514825374 shutdown_connections 2024-09-19T03:58:20.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.747+0000 7f0dde37b640 1 --2- 172.21.15.196:0/1514825374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 0x7f0dc0008610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:20.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.747+0000 7f0dde37b640 1 -- 172.21.15.196:0/1514825374 >> 172.21.15.196:0/1514825374 conn(0x7f0dd80fc280 msgr2=0x7f0dd80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:20.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.747+0000 7f0dde37b640 1 -- 172.21.15.196:0/1514825374 shutdown_connections 2024-09-19T03:58:20.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.747+0000 7f0dde37b640 1 -- 172.21.15.196:0/1514825374 wait complete. 2024-09-19T03:58:20.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.748+0000 7f0dde37b640 1 Processor -- start 2024-09-19T03:58:20.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.748+0000 7f0dde37b640 1 -- start start 2024-09-19T03:58:20.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dde37b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 0x7f0dd81004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:20.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dde37b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 0x7f0dd8100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:20.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dde37b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd81097a0 con 0x7f0dc00061a0 2024-09-19T03:58:20.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dde37b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8109620 con 0x7f0dd81009e0 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 0x7f0dd8100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 0x7f0dd8100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38378/0 (socket says 172.21.15.196:38378) 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd77fe640 1 -- 172.21.15.196:0/2747213706 learned_addr learned my addr 172.21.15.196:0/2747213706 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd7fff640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 0x7f0dd81004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd77fe640 1 -- 172.21.15.196:0/2747213706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 msgr2=0x7f0dd8100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd77fe640 1 -- 172.21.15.196:0/2747213706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 msgr2=0x7f0dd8100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd77fe640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 0x7f0dd8100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd77fe640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 0x7f0dd8100e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd7fff640 1 -- 172.21.15.196:0/2747213706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 msgr2=0x7f0dd8100e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd7fff640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 0x7f0dd8100e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.749+0000 7f0dd7fff640 1 -- 172.21.15.196:0/2747213706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd8102390 con 0x7f0dc00061a0 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.750+0000 7f0dd7fff640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 0x7f0dd81004a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8069f20 tx=0x7f0dc8033630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:20.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.750+0000 7f0dd57fa640 1 -- 172.21.15.196:0/2747213706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0dc8038040 con 0x7f0dc00061a0 2024-09-19T03:58:20.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.750+0000 7f0dd57fa640 1 -- 172.21.15.196:0/2747213706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc8005530 con 0x7f0dc00061a0 2024-09-19T03:58:20.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.750+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd81a3630 con 0x7f0dc00061a0 2024-09-19T03:58:20.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.750+0000 7f0dd57fa640 1 -- 172.21.15.196:0/2747213706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0dc8004820 con 0x7f0dc00061a0 2024-09-19T03:58:20.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.750+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd81a3b60 con 0x7f0dc00061a0 2024-09-19T03:58:20.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.751+0000 7f0dd57fa640 1 -- 172.21.15.196:0/2747213706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0dc8043020 con 0x7f0dc00061a0 2024-09-19T03:58:20.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.751+0000 7f0dd57fa640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0dc4039090 0x7f0dc403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:20.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.751+0000 7f0dd57fa640 1 -- 172.21.15.196:0/2747213706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0dc806f570 con 0x7f0dc00061a0 2024-09-19T03:58:20.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.751+0000 7f0dd77fe640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0dc4039090 0x7f0dc403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:20.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.751+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd806a4f0 con 0x7f0dc00061a0 2024-09-19T03:58:20.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.752+0000 7f0dd77fe640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0dc4039090 0x7f0dc403b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009100 tx=0x7f0dcc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:20.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.757+0000 7f0dd57fa640 1 -- 172.21.15.196:0/2747213706 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0dc804a020 con 0x7f0dc00061a0 2024-09-19T03:58:20.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.988+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0dd81019d0 con 0x7f0dc00061a0 2024-09-19T03:58:20.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.989+0000 7f0dd57fa640 1 -- 172.21.15.196:0/2747213706 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0dc804cbc0 con 0x7f0dc00061a0 2024-09-19T03:58:20.990 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:58:20.991 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","modified":"2024-09-19T03:58:14.249876Z","created":"2024-09-19T03:56:39.538905Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T03:58:20.991 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0dc4039090 msgr2=0x7f0dc403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0dc4039090 0x7f0dc403b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009100 tx=0x7f0dcc008040 comp rx=0 tx=0).stop 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 msgr2=0x7f0dd81004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 0x7f0dd81004a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8069f20 tx=0x7f0dc8033630 comp rx=0 tx=0).stop 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 shutdown_connections 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0dc4039090 0x7f0dc403b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd81009e0 0x7f0dd8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 --2- 172.21.15.196:0/2747213706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00061a0 0x7f0dd81004a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 >> 172.21.15.196:0/2747213706 conn(0x7f0dd80fc280 msgr2=0x7f0dd818fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 shutdown_connections 2024-09-19T03:58:20.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:20.991+0000 7f0dde37b640 1 -- 172.21.15.196:0/2747213706 wait complete. 2024-09-19T03:58:21.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:21 smithi196 ceph-mon[35731]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:21.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:21 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/2747213706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T03:58:21.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:21 smithi019 ceph-mon[30938]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:21.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:21 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/2747213706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T03:58:22.562 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T03:58:22.562 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T03:58:22.818 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T03:58:23.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:23 smithi111 systemd[1]: Started Ceph mon.b for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T03:58:28.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T03:58:28.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: mon.a calling monitor election 2024-09-19T03:58:28.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: mon.c calling monitor election 2024-09-19T03:58:28.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:28.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: mon.b calling monitor election 2024-09-19T03:58:28.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T03:58:28.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: monmap epoch 3 2024-09-19T03:58:28.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:28.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: last_changed 2024-09-19T03:58:22.989439+0000 2024-09-19T03:58:28.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: created 2024-09-19T03:56:39.538905+0000 2024-09-19T03:58:28.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: min_mon_release 19 (squid) 2024-09-19T03:58:28.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: election_strategy: 1 2024-09-19T03:58:28.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-19T03:58:28.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T03:58:28.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-19T03:58:28.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: fsmap 2024-09-19T03:58:28.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T03:58:28.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: mgrmap e13: x(active, since 47s) 2024-09-19T03:58:28.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: overall HEALTH_OK 2024-09-19T03:58:28.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: mon.a calling monitor election 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: mon.c calling monitor election 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: mon.b calling monitor election 2024-09-19T03:58:28.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: monmap epoch 3 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: last_changed 2024-09-19T03:58:22.989439+0000 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: created 2024-09-19T03:56:39.538905+0000 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: min_mon_release 19 (squid) 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: election_strategy: 1 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-19T03:58:28.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: fsmap 2024-09-19T03:58:28.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T03:58:28.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: mgrmap e13: x(active, since 47s) 2024-09-19T03:58:28.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: overall HEALTH_OK 2024-09-19T03:58:28.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: mon.a calling monitor election 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: mon.c calling monitor election 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: mon.b calling monitor election 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: monmap epoch 3 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: last_changed 2024-09-19T03:58:22.989439+0000 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: created 2024-09-19T03:56:39.538905+0000 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: min_mon_release 19 (squid) 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: election_strategy: 1 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T03:58:28.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-19T03:58:28.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: fsmap 2024-09-19T03:58:28.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T03:58:28.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: mgrmap e13: x(active, since 47s) 2024-09-19T03:58:28.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: overall HEALTH_OK 2024-09-19T03:58:28.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:28.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:29.325 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:29 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:29.325 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:29 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:29.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.322+0000 7f719a615640 1 -- 172.21.15.196:0/2709000034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 msgr2=0x7f717c0065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:29.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.322+0000 7f719a615640 1 --2- 172.21.15.196:0/2709000034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 0x7f717c0065f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f71941027e0 tx=0x7f718402c4a0 comp rx=0 tx=0).stop 2024-09-19T03:58:29.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.322+0000 7f719a615640 1 -- 172.21.15.196:0/2709000034 shutdown_connections 2024-09-19T03:58:29.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.322+0000 7f719a615640 1 --2- 172.21.15.196:0/2709000034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 0x7f717c0065f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:29.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.322+0000 7f719a615640 1 -- 172.21.15.196:0/2709000034 >> 172.21.15.196:0/2709000034 conn(0x7f71940fd330 msgr2=0x7f71940ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:29.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.322+0000 7f719a615640 1 -- 172.21.15.196:0/2709000034 shutdown_connections 2024-09-19T03:58:29.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.322+0000 7f719a615640 1 -- 172.21.15.196:0/2709000034 wait complete. 2024-09-19T03:58:29.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f719a615640 1 Processor -- start 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f719a615640 1 -- start start 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f719a615640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 0x7f71940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f719a615640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7194074a30 0x7f7194074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f719a615640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71941a3700 0x7f71941a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f719a615640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7194109710 con 0x7f717c006210 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f719a615640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7194109590 con 0x7f71941a3700 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f719a615640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7194109890 con 0x7f7194074a30 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f71937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7194074a30 0x7f7194074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f7193fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 0x7f71940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f71937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7194074a30 0x7f7194074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38388/0 (socket says 172.21.15.196:38388) 2024-09-19T03:58:29.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.323+0000 7f71937fe640 1 -- 172.21.15.196:0/2223074473 learned_addr learned my addr 172.21.15.196:0/2223074473 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f7198bb2640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71941a3700 0x7f71941a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f7193fff640 1 -- 172.21.15.196:0/2223074473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7194074a30 msgr2=0x7f7194074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f7193fff640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7194074a30 0x7f7194074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f7193fff640 1 -- 172.21.15.196:0/2223074473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71941a3700 msgr2=0x7f71941a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f7193fff640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71941a3700 0x7f71941a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f7193fff640 1 -- 172.21.15.196:0/2223074473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71941a61c0 con 0x7f717c006210 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f7198bb2640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71941a3700 0x7f71941a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f7193fff640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 0x7f71940744f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71941027e0 tx=0x7f7184033630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f71917fa640 1 -- 172.21.15.196:0/2223074473 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184041020 con 0x7f717c006210 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f71917fa640 1 -- 172.21.15.196:0/2223074473 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718403d480 con 0x7f717c006210 2024-09-19T03:58:29.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71941a6450 con 0x7f717c006210 2024-09-19T03:58:29.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f71917fa640 1 -- 172.21.15.196:0/2223074473 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184038040 con 0x7f717c006210 2024-09-19T03:58:29.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.324+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71941a6980 con 0x7f717c006210 2024-09-19T03:58:29.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.325+0000 7f71917fa640 1 -- 172.21.15.196:0/2223074473 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7184044020 con 0x7f717c006210 2024-09-19T03:58:29.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.325+0000 7f71917fa640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7180039730 0x7f718003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:29.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.325+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7194075a20 con 0x7f717c006210 2024-09-19T03:58:29.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.325+0000 7f71917fa640 1 -- 172.21.15.196:0/2223074473 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f718406eae0 con 0x7f717c006210 2024-09-19T03:58:29.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.326+0000 7f71937fe640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7180039730 0x7f718003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:29.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.326+0000 7f71937fe640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7180039730 0x7f718003bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7188009b80 tx=0x7f7188009240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:29.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.331+0000 7f71917fa640 1 -- 172.21.15.196:0/2223074473 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f718404b020 con 0x7f717c006210 2024-09-19T03:58:29.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:29 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:29.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:29 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:29.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:29 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:29.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:29 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T03:58:29.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.573+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f719402d7c0 con 0x7f717c006210 2024-09-19T03:58:29.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.574+0000 7f71917fa640 1 -- 172.21.15.196:0/2223074473 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f718403d630 con 0x7f717c006210 2024-09-19T03:58:29.577 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:58:29.577 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","modified":"2024-09-19T03:58:22.989439Z","created":"2024-09-19T03:56:39.538905Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-19T03:58:29.577 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-09-19T03:58:29.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7180039730 msgr2=0x7f718003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:29.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7180039730 0x7f718003bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7188009b80 tx=0x7f7188009240 comp rx=0 tx=0).stop 2024-09-19T03:58:29.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 msgr2=0x7f71940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:29.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 0x7f71940744f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71941027e0 tx=0x7f7184033630 comp rx=0 tx=0).stop 2024-09-19T03:58:29.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 shutdown_connections 2024-09-19T03:58:29.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7180039730 0x7f718003bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:29.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71941a3700 0x7f71941a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:29.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7194074a30 0x7f7194074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:29.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 --2- 172.21.15.196:0/2223074473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717c006210 0x7f71940744f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:29.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 >> 172.21.15.196:0/2223074473 conn(0x7f71940fd330 msgr2=0x7f7194106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:29.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 shutdown_connections 2024-09-19T03:58:29.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:29.576+0000 7f719a615640 1 -- 172.21.15.196:0/2223074473 wait complete. 2024-09-19T03:58:30.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:30.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:30.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/2223074473' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.163 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.163 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.163 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:30.163 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:30.163 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:30.188 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T03:58:30.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T03:58:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/2223074473' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:30.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:30.407 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:30.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:30.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:30.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-19T03:58:30.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/2223074473' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:30.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:30.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:30.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:31.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:31 smithi019 ceph-mon[30938]: Updating smithi111:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:31 smithi019 ceph-mon[30938]: Updating smithi196:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:31 smithi019 ceph-mon[30938]: Updating smithi019:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:31 smithi019 ceph-mon[30938]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T03:58:31.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:31 smithi019 ceph-mon[30938]: Reconfiguring daemon mon.a on smithi019 2024-09-19T03:58:31.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:58:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:31 smithi111 ceph-mon[35365]: Updating smithi111:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:31 smithi111 ceph-mon[35365]: Updating smithi196:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:31 smithi111 ceph-mon[35365]: Updating smithi019:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:31 smithi111 ceph-mon[35365]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T03:58:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:31 smithi111 ceph-mon[35365]: Reconfiguring daemon mon.a on smithi019 2024-09-19T03:58:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:31 smithi196 ceph-mon[35731]: Updating smithi111:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:31 smithi196 ceph-mon[35731]: Updating smithi196:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:31 smithi196 ceph-mon[35731]: Updating smithi019:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T03:58:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:31 smithi196 ceph-mon[35731]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T03:58:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:31 smithi196 ceph-mon[35731]: Reconfiguring daemon mon.a on smithi019 2024-09-19T03:58:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3891112157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 msgr2=0x7fa0bc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3891112157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 0x7fa0bc10e610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009ca0 tx=0x7fa0b402a950 comp rx=0 tx=0).stop 2024-09-19T03:58:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3891112157 shutdown_connections 2024-09-19T03:58:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3891112157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 0x7fa0bc10e610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3891112157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc103660 0x7fa0bc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3891112157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0bc102d40 0x7fa0bc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3891112157 >> 172.21.15.19:0/3891112157 conn(0x7fa0bc0fc7a0 msgr2=0x7fa0bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3891112157 shutdown_connections 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.805+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3891112157 wait complete. 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0c18a3640 1 Processor -- start 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0c18a3640 1 -- start start 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0c18a3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc102d40 0x7fa0bc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0c18a3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0bc103660 0x7fa0bc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0c18a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 0x7fa0bc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0c18a3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc111f70 con 0x7fa0bc10c170 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0c18a3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc111df0 con 0x7fa0bc103660 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0c18a3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1120f0 con 0x7fa0bc102d40 2024-09-19T03:58:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 0x7fa0bc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 0x7fa0bc101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53828/0 (socket says 172.21.15.19:53828) 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0bb7fe640 1 -- 172.21.15.19:0/3471248864 learned_addr learned my addr 172.21.15.19:0/3471248864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0baffd640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc102d40 0x7fa0bc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.806+0000 7fa0b3fff640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0bc103660 0x7fa0bc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0bb7fe640 1 -- 172.21.15.19:0/3471248864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc102d40 msgr2=0x7fa0bc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0bb7fe640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc102d40 0x7fa0bc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0bb7fe640 1 -- 172.21.15.19:0/3471248864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0bc103660 msgr2=0x7fa0bc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0bb7fe640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0bc103660 0x7fa0bc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0bb7fe640 1 -- 172.21.15.19:0/3471248864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc1a67f0 con 0x7fa0bc10c170 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0bb7fe640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 0x7fa0bc101420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006010 tx=0x7fa0b402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0c08a1640 1 -- 172.21.15.19:0/3471248864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4034040 con 0x7fa0bc10c170 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0c08a1640 1 -- 172.21.15.19:0/3471248864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b4003e00 con 0x7fa0bc10c170 2024-09-19T03:58:31.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0bc1a6a80 con 0x7fa0bc10c170 2024-09-19T03:58:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0c08a1640 1 -- 172.21.15.19:0/3471248864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40456e0 con 0x7fa0bc10c170 2024-09-19T03:58:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.807+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0bc1a6fb0 con 0x7fa0bc10c170 2024-09-19T03:58:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.808+0000 7fa0c08a1640 1 -- 172.21.15.19:0/3471248864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa0b4037020 con 0x7fa0bc10c170 2024-09-19T03:58:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.808+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0bc103120 con 0x7fa0bc10c170 2024-09-19T03:58:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.808+0000 7fa0c08a1640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa0a80394d0 0x7fa0a803b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.808+0000 7fa0c08a1640 1 -- 172.21.15.19:0/3471248864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa0b406d810 con 0x7fa0bc10c170 2024-09-19T03:58:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.808+0000 7fa0baffd640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa0a80394d0 0x7fa0a803b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.809+0000 7fa0baffd640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa0a80394d0 0x7fa0a803b990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0058c0 tx=0x7fa0ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.813+0000 7fa0c08a1640 1 -- 172.21.15.19:0/3471248864 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0b4004160 con 0x7fa0bc10c170 2024-09-19T03:58:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.988+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa0bc1a71e0 con 0x7fa0bc10c170 2024-09-19T03:58:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.989+0000 7fa0c08a1640 1 -- 172.21.15.19:0/3471248864 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fa0b4039c80 con 0x7fa0bc10c170 2024-09-19T03:58:31.990 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:31.990 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-09-19T03:58:31.990 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T03:58:31.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-19T03:58:31.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.990+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa0a80394d0 msgr2=0x7fa0a803b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:31.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.990+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa0a80394d0 0x7fa0a803b990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0058c0 tx=0x7fa0ac00a2d0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.990+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 msgr2=0x7fa0bc101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:31.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.990+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 0x7fa0bc101420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006010 tx=0x7fa0b402f710 comp rx=0 tx=0).stop 2024-09-19T03:58:31.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.991+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 shutdown_connections 2024-09-19T03:58:31.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.991+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa0a80394d0 0x7fa0a803b990 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.991+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0bc10c170 0x7fa0bc101420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.991+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0bc103660 0x7fa0bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.991+0000 7fa0c18a3640 1 --2- 172.21.15.19:0/3471248864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc102d40 0x7fa0bc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.991+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 >> 172.21.15.19:0/3471248864 conn(0x7fa0bc0fc7a0 msgr2=0x7fa0bc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.991+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 shutdown_connections 2024-09-19T03:58:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:31.991+0000 7fa0c18a3640 1 -- 172.21.15.19:0/3471248864 wait complete. 2024-09-19T03:58:32.285 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:32 smithi019 ceph-mon[30938]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:32.285 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:32.285 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:32.285 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:32.285 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:32.285 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:32.285 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:32 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3471248864' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:32 smithi111 ceph-mon[35365]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:32 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3471248864' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:32 smithi196 ceph-mon[35731]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:32 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3471248864' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:32.656 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T03:58:32.656 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:58:32.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T03:58:32.686 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:58:32.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:58:32.754 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:58:32.754 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T03:58:32.782 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:58:32.782 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:58:32.847 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:58:32.847 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T03:58:32.874 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:58:32.874 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T03:58:32.939 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-09-19T03:58:32.940 INFO:tasks.cephadm:Adding mgr.y on smithi111 2024-09-19T03:58:32.940 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-09-19T03:58:32.940 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi111=y;smithi196=z' 2024-09-19T03:58:33.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:33 smithi196 ceph-mon[35731]: Reconfiguring mon.b (monmap changed)... 2024-09-19T03:58:33.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:33 smithi196 ceph-mon[35731]: Reconfiguring daemon mon.b on smithi111 2024-09-19T03:58:33.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:33.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:33.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:33.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:33.162 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:33.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:33 smithi111 ceph-mon[35365]: Reconfiguring mon.b (monmap changed)... 2024-09-19T03:58:33.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:33 smithi111 ceph-mon[35365]: Reconfiguring daemon mon.b on smithi111 2024-09-19T03:58:33.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:33.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:33.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:33.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:33.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:33.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:33 smithi019 ceph-mon[30938]: Reconfiguring mon.b (monmap changed)... 2024-09-19T03:58:33.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:33 smithi019 ceph-mon[30938]: Reconfiguring daemon mon.b on smithi111 2024-09-19T03:58:33.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:33.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:33.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T03:58:33.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T03:58:33.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:33.776 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T03:58:34.183 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:34 smithi196 ceph-mon[35731]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:34.183 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:34 smithi196 ceph-mon[35731]: Reconfiguring mon.c (monmap changed)... 2024-09-19T03:58:34.183 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:34 smithi196 ceph-mon[35731]: Reconfiguring daemon mon.c on smithi196 2024-09-19T03:58:34.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.181+0000 7f34e3e54640 1 -- 172.21.15.196:0/539752920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1016e0 msgr2=0x7f34dc101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:34.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.181+0000 7f34e3e54640 1 --2- 172.21.15.196:0/539752920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1016e0 0x7f34dc101ae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009880 tx=0x7f34d802a5d0 comp rx=0 tx=0).stop 2024-09-19T03:58:34.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.181+0000 7f34e3e54640 1 -- 172.21.15.196:0/539752920 shutdown_connections 2024-09-19T03:58:34.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.181+0000 7f34e3e54640 1 --2- 172.21.15.196:0/539752920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1016e0 0x7f34dc101ae0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:34.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.181+0000 7f34e3e54640 1 -- 172.21.15.196:0/539752920 >> 172.21.15.196:0/539752920 conn(0x7f34dc0fce90 msgr2=0x7f34dc0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:34.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.181+0000 7f34e3e54640 1 -- 172.21.15.196:0/539752920 shutdown_connections 2024-09-19T03:58:34.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.181+0000 7f34e3e54640 1 -- 172.21.15.196:0/539752920 wait complete. 2024-09-19T03:58:34.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.182+0000 7f34e3e54640 1 Processor -- start 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.182+0000 7f34e3e54640 1 -- start start 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.182+0000 7f34e3e54640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34dc1016e0 0x7f34dc196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.182+0000 7f34e3e54640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1974f0 0x7f34dc19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.182+0000 7f34e3e54640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34dc197970 0x7f34dc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e3e54640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1077f0 con 0x7f34dc1974f0 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e3e54640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc107670 con 0x7f34dc1016e0 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e3e54640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc107970 con 0x7f34dc197970 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34dc197970 0x7f34dc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34dc197970 0x7f34dc19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47072/0 (socket says 172.21.15.196:47072) 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 -- 172.21.15.196:0/3625446678 learned_addr learned my addr 172.21.15.196:0/3625446678 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e13ef640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1974f0 0x7f34dc19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e1bf0640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34dc1016e0 0x7f34dc196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:34.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 -- 172.21.15.196:0/3625446678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34dc1016e0 msgr2=0x7f34dc196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34dc1016e0 0x7f34dc196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 -- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1974f0 msgr2=0x7f34dc19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1974f0 0x7f34dc19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 -- 172.21.15.196:0/3625446678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc19f3b0 con 0x7f34dc197970 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e13ef640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1974f0 0x7f34dc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e23f1640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34dc197970 0x7f34dc19eaf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f34d400ba50 tx=0x7f34d400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34d2ffd640 1 -- 172.21.15.196:0/3625446678 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4019070 con 0x7f34dc197970 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34d2ffd640 1 -- 172.21.15.196:0/3625446678 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d400cbe0 con 0x7f34dc197970 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc19f6a0 con 0x7f34dc197970 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.183+0000 7f34d2ffd640 1 -- 172.21.15.196:0/3625446678 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40076a0 con 0x7f34dc197970 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.184+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc19fc60 con 0x7f34dc197970 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.184+0000 7f34d2ffd640 1 -- 172.21.15.196:0/3625446678 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f34d401c070 con 0x7f34dc197970 2024-09-19T03:58:34.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.185+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34dc1905b0 con 0x7f34dc197970 2024-09-19T03:58:34.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.185+0000 7f34d2ffd640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34c8039730 0x7f34c803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:34.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.185+0000 7f34d2ffd640 1 -- 172.21.15.196:0/3625446678 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f34d404be30 con 0x7f34dc197970 2024-09-19T03:58:34.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.185+0000 7f34e1bf0640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34c8039730 0x7f34c803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:34.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.185+0000 7f34e1bf0640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34c8039730 0x7f34c803bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f34d802abc0 tx=0x7f34d8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:34.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.190+0000 7f34d2ffd640 1 -- 172.21.15.196:0/3625446678 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34d401d4f0 con 0x7f34dc197970 2024-09-19T03:58:34.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.375+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi111=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7f34dc0626c0 con 0x7f34c8039730 2024-09-19T03:58:34.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.382+0000 7f34d2ffd640 1 -- 172.21.15.196:0/3625446678 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f34dc0626c0 con 0x7f34c8039730 2024-09-19T03:58:34.383 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34c8039730 msgr2=0x7f34c803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34c8039730 0x7f34c803bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f34d802abc0 tx=0x7f34d8031400 comp rx=0 tx=0).stop 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34dc197970 msgr2=0x7f34dc19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34dc197970 0x7f34dc19eaf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f34d400ba50 tx=0x7f34d400bf10 comp rx=0 tx=0).stop 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 shutdown_connections 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34c8039730 0x7f34c803bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34dc197970 0x7f34dc19eaf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34dc1974f0 0x7f34dc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:34.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 --2- 172.21.15.196:0/3625446678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34dc1016e0 0x7f34dc196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:34.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.384+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 >> 172.21.15.196:0/3625446678 conn(0x7f34dc0fce90 msgr2=0x7f34dc0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:34.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.385+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 shutdown_connections 2024-09-19T03:58:34.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:58:34.385+0000 7f34e3e54640 1 -- 172.21.15.196:0/3625446678 wait complete. 2024-09-19T03:58:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:34 smithi111 ceph-mon[35365]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:34 smithi111 ceph-mon[35365]: Reconfiguring mon.c (monmap changed)... 2024-09-19T03:58:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:34 smithi111 ceph-mon[35365]: Reconfiguring daemon mon.c on smithi196 2024-09-19T03:58:34.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:34 smithi019 ceph-mon[30938]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:34.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:34 smithi019 ceph-mon[30938]: Reconfiguring mon.c (monmap changed)... 2024-09-19T03:58:34.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:34 smithi019 ceph-mon[30938]: Reconfiguring daemon mon.c on smithi196 2024-09-19T03:58:35.204 DEBUG:teuthology.orchestra.run.smithi111:mgr.y> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T03:58:35.207 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T03:58:35.210 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T03:58:35.211 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T03:58:35.211 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T03:58:35.232 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T03:58:35.232 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-09-19T03:58:35.291 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T03:58:35.291 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:35.291 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T03:58:35.291 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:35.292 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:35.292 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-19 03:57:51.024085284 +0000 2024-09-19T03:58:35.292 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-19 03:53:05.134315926 +0000 2024-09-19T03:58:35.292 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-19 03:53:05.134315926 +0000 2024-09-19T03:58:35.292 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-19 03:53:05.134315926 +0000 2024-09-19T03:58:35.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T03:58:35.361 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-19T03:58:35.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-19T03:58:35.362 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.0002311 s, 2.2 MB/s 2024-09-19T03:58:35.363 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T03:58:35.415 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:35.415 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi111=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:35.415 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: Saving service mgr spec with placement smithi019=x;smithi111=y;smithi196=z;count:3 2024-09-19T03:58:35.415 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:35.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:35.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:35 smithi196 ceph-mon[35731]: Deploying daemon mgr.z on smithi196 2024-09-19T03:58:35.423 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-19 03:57:51.052085016 +0000 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-19 03:53:05.471312807 +0000 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-19 03:53:05.471312807 +0000 2024-09-19T03:58:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-19 03:53:05.471312807 +0000 2024-09-19T03:58:35.483 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T03:58:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-19T03:58:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-19T03:58:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000201165 s, 2.5 MB/s 2024-09-19T03:58:35.550 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T03:58:35.610 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-09-19T03:58:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi111=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: Saving service mgr spec with placement smithi019=x;smithi111=y;smithi196=z;count:3 2024-09-19T03:58:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:35.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:35.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:35.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T03:58:35.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:35.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:35.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:35 smithi111 ceph-mon[35365]: Deploying daemon mgr.z on smithi196 2024-09-19T03:58:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T03:58:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:35.669 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T03:58:35.669 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:35.670 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:35.670 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-19 03:57:51.079084758 +0000 2024-09-19T03:58:35.670 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-19 03:53:05.810309671 +0000 2024-09-19T03:58:35.670 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-19 03:53:05.810309671 +0000 2024-09-19T03:58:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-19 03:53:05.810309671 +0000 2024-09-19T03:58:35.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T03:58:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi111=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: Saving service mgr spec with placement smithi019=x;smithi111=y;smithi196=z;count:3 2024-09-19T03:58:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:35.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:35.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:35.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:35.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:35.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T03:58:35.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:35.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:35.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:35 smithi019 ceph-mon[30938]: Deploying daemon mgr.z on smithi196 2024-09-19T03:58:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-19T03:58:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-19T03:58:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000231907 s, 2.2 MB/s 2024-09-19T03:58:35.709 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T03:58:35.768 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-09-19T03:58:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T03:58:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:35.825 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T03:58:35.825 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:35.825 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:35.825 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-19 03:57:51.106084499 +0000 2024-09-19T03:58:35.825 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-19 03:53:06.134306673 +0000 2024-09-19T03:58:35.825 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-19 03:53:06.134306673 +0000 2024-09-19T03:58:35.826 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-19 03:53:06.134306673 +0000 2024-09-19T03:58:35.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T03:58:35.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-19T03:58:35.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-19T03:58:35.891 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000264113 s, 1.9 MB/s 2024-09-19T03:58:35.892 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T03:58:35.950 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-19T03:58:35.950 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T03:58:35.970 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T03:58:35.971 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-09-19T03:58:36.029 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T03:58:36.029 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:36.029 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T03:58:36.029 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:36.029 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:36.029 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-19 03:58:22.426547807 +0000 2024-09-19T03:58:36.029 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-19 03:53:23.771326126 +0000 2024-09-19T03:58:36.029 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-19 03:53:23.771326126 +0000 2024-09-19T03:58:36.030 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-19 03:53:23.771326126 +0000 2024-09-19T03:58:36.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T03:58:36.097 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-19T03:58:36.097 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-19T03:58:36.097 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000358151 s, 1.4 MB/s 2024-09-19T03:58:36.098 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T03:58:36.157 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-09-19T03:58:36.214 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T03:58:36.214 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:36.214 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T03:58:36.214 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:36.214 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:36.215 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-19 03:58:22.426547807 +0000 2024-09-19T03:58:36.215 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-19 03:53:24.089320484 +0000 2024-09-19T03:58:36.215 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-19 03:53:24.089320484 +0000 2024-09-19T03:58:36.215 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-19 03:53:24.089320484 +0000 2024-09-19T03:58:36.215 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T03:58:36.280 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-19T03:58:36.280 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-19T03:58:36.280 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000228738 s, 2.2 MB/s 2024-09-19T03:58:36.281 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T03:58:36.338 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-09-19T03:58:36.395 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T03:58:36.396 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:36.396 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-19T03:58:36.396 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:36.396 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:36.396 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-19 03:58:22.426547807 +0000 2024-09-19T03:58:36.396 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-19 03:53:24.407314843 +0000 2024-09-19T03:58:36.396 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-19 03:53:24.407314843 +0000 2024-09-19T03:58:36.396 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-19 03:53:24.407314843 +0000 2024-09-19T03:58:36.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T03:58:36.460 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-19T03:58:36.461 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-19T03:58:36.461 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000223632 s, 2.3 MB/s 2024-09-19T03:58:36.462 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T03:58:36.519 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-09-19T03:58:36.574 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T03:58:36.575 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:36.575 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T03:58:36.575 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:36.575 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:36.575 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-19 03:58:22.427547789 +0000 2024-09-19T03:58:36.575 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-19 03:53:24.718309326 +0000 2024-09-19T03:58:36.575 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-19 03:53:24.718309326 +0000 2024-09-19T03:58:36.575 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-19 03:53:24.718309326 +0000 2024-09-19T03:58:36.575 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T03:58:36.639 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-19T03:58:36.639 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-19T03:58:36.639 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000200123 s, 2.6 MB/s 2024-09-19T03:58:36.640 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T03:58:36.697 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T03:58:36.697 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T03:58:36.713 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T03:58:36.713 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-09-19T03:58:36.769 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T03:58:36.769 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:36.769 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T03:58:36.770 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:36.770 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:36.770 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-19 03:58:36.682822200 +0000 2024-09-19T03:58:36.770 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-19 03:53:18.726118928 +0000 2024-09-19T03:58:36.770 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-19 03:53:18.726118928 +0000 2024-09-19T03:58:36.770 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-19 03:53:18.726118928 +0000 2024-09-19T03:58:36.770 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T03:58:36.833 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:36 smithi196 systemd[1]: Starting Ceph mgr.z for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T03:58:36.837 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-19T03:58:36.837 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-19T03:58:36.837 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000288384 s, 1.8 MB/s 2024-09-19T03:58:36.838 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T03:58:36.896 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-19 03:58:36.682822200 +0000 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-19 03:53:19.050111760 +0000 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-19 03:53:19.050111760 +0000 2024-09-19T03:58:36.952 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-19 03:53:19.050111760 +0000 2024-09-19T03:58:36.953 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T03:58:37.016 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-19T03:58:37.016 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-19T03:58:37.016 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000211775 s, 2.4 MB/s 2024-09-19T03:58:37.017 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T03:58:37.073 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-09-19T03:58:37.111 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:36 smithi196 podman[37202]: 2024-09-19 03:58:36.861769577 +0000 UTC m=+0.015890296 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T03:58:37.112 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:36 smithi196 podman[37202]: 2024-09-19 03:58:36.979957209 +0000 UTC m=+0.134077909 container create 1dfea3d0731125a8246c6c0e5e3f3e8f61f6f82181a4a57b3600ed58b693d834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T03:58:37.126 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T03:58:37.126 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:37.127 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T03:58:37.127 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:37.127 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:37.127 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-19 03:58:36.682822200 +0000 2024-09-19T03:58:37.127 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-19 03:53:19.376104548 +0000 2024-09-19T03:58:37.127 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-19 03:53:19.376104548 +0000 2024-09-19T03:58:37.127 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-19 03:53:19.376104548 +0000 2024-09-19T03:58:37.127 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T03:58:37.190 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-19T03:58:37.190 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-19T03:58:37.191 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000217514 s, 2.4 MB/s 2024-09-19T03:58:37.192 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T03:58:37.248 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-09-19T03:58:37.305 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T03:58:37.305 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T03:58:37.305 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T03:58:37.306 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T03:58:37.306 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T03:58:37.306 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-19 03:58:36.683822179 +0000 2024-09-19T03:58:37.306 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-19 03:53:19.683097756 +0000 2024-09-19T03:58:37.306 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-19 03:53:19.683097756 +0000 2024-09-19T03:58:37.306 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-19 03:53:19.683097756 +0000 2024-09-19T03:58:37.306 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T03:58:37.370 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-19T03:58:37.371 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-19T03:58:37.371 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000195045 s, 2.6 MB/s 2024-09-19T03:58:37.372 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T03:58:37.428 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-09-19T03:58:37.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T03:58:37.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mon[35731]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:37.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:37.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T03:58:37.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:37.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:37.459 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 podman[37202]: 2024-09-19 03:58:37.199625334 +0000 UTC m=+0.353746033 container init 1dfea3d0731125a8246c6c0e5e3f3e8f61f6f82181a4a57b3600ed58b693d834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T03:58:37.459 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 podman[37202]: 2024-09-19 03:58:37.202314679 +0000 UTC m=+0.356435379 container start 1dfea3d0731125a8246c6c0e5e3f3e8f61f6f82181a4a57b3600ed58b693d834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T03:58:37.459 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-mgr[37327]: -- 172.21.15.196:0/2987161492 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556971c5b0e0 con 0x556971c38800 2024-09-19T03:58:37.459 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 bash[37202]: 1dfea3d0731125a8246c6c0e5e3f3e8f61f6f82181a4a57b3600ed58b693d834 2024-09-19T03:58:37.459 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 systemd[1]: Started Ceph mgr.z for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T03:58:37.460 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:37.327+0000 7f51db687040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T03:58:37.460 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:37.389+0000 7f51db687040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T03:58:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:37 smithi111 ceph-mon[35365]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:37 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:37 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:37 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:37 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:37 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T03:58:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:37 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:37.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:37 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:37 smithi019 ceph-mon[30938]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:37 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:37 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:37 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:37 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:37 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T03:58:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:37 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:37.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:37 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:58:37.889 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:37.540+0000 7f51db687040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T03:58:37.889 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:37.605+0000 7f51db687040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T03:58:37.889 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:37.750+0000 7f51db687040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:58:37.889 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:37.887+0000 7f51db687040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T03:58:38.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:38 smithi196 ceph-mon[35731]: Deploying daemon mgr.y on smithi111 2024-09-19T03:58:38.458 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:38.198+0000 7f51db687040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T03:58:38.458 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:38.259+0000 7f51db687040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T03:58:38.458 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:38.444+0000 7f51db687040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T03:58:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:38 smithi111 ceph-mon[35365]: Deploying daemon mgr.y on smithi111 2024-09-19T03:58:38.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:38 smithi019 ceph-mon[30938]: Deploying daemon mgr.y on smithi111 2024-09-19T03:58:39.394 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 podman[36714]: 2024-09-19 03:58:39.192847956 +0000 UTC m=+0.015841682 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T03:58:39.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:39 smithi111 ceph-mon[35365]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:39.655 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 podman[36714]: 2024-09-19 03:58:39.494738227 +0000 UTC m=+0.317731935 container init 539ccb18f7b0082ff31a9965c240f9b9f05adda35f79dfc5038524c546d639ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T03:58:39.655 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 podman[36714]: 2024-09-19 03:58:39.497957118 +0000 UTC m=+0.320950826 container start 539ccb18f7b0082ff31a9965c240f9b9f05adda35f79dfc5038524c546d639ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T03:58:39.655 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 ceph-mgr[36733]: -- 172.21.15.111:0/3653810992 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f95a010e0 con 0x564f959de800 2024-09-19T03:58:39.655 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 bash[36714]: 539ccb18f7b0082ff31a9965c240f9b9f05adda35f79dfc5038524c546d639ea 2024-09-19T03:58:39.655 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 systemd[1]: Started Ceph mgr.y for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T03:58:39.655 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:39.623+0000 7fb3f9cf8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T03:58:39.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:39 smithi019 ceph-mon[30938]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:39.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:39 smithi196 ceph-mon[35731]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:39.708 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:39 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:39.324+0000 7f51db687040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T03:58:39.708 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:39 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:39.390+0000 7f51db687040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T03:58:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:58:40.059 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-09-19T03:58:40.077 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:39.692+0000 7fb3f9cf8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T03:58:40.077 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:39.848+0000 7fb3f9cf8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T03:58:40.077 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:39.917+0000 7fb3f9cf8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T03:58:40.077 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:40.075+0000 7fb3f9cf8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:58:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:58:40.358 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:40.226+0000 7fb3f9cf8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T03:58:40.418 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.152+0000 7f51db687040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T03:58:40.418 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.270+0000 7f51db687040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T03:58:40.418 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.416+0000 7f51db687040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:58:40.617 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:40.552+0000 7fb3f9cf8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T03:58:40.617 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:40.615+0000 7fb3f9cf8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T03:58:40.708 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.482+0000 7f51db687040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T03:58:40.709 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.637+0000 7f51db687040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T03:58:40.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:40.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:40.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 -- 172.21.15.19:0/3788840285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 msgr2=0x7f8914108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3788840285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 0x7f8914108f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f890c009900 tx=0x7f890c02a410 comp rx=0 tx=0).stop 2024-09-19T03:58:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 -- 172.21.15.19:0/3788840285 shutdown_connections 2024-09-19T03:58:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3788840285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 0x7f8914108f20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3788840285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914101cd0 0x7f8914102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3788840285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8914100ad0 0x7f8914100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 -- 172.21.15.19:0/3788840285 >> 172.21.15.19:0/3788840285 conn(0x7f89140fc280 msgr2=0x7f89140fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:58:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 -- 172.21.15.19:0/3788840285 shutdown_connections 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.739+0000 7f891b2d9640 1 -- 172.21.15.19:0/3788840285 wait complete. 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f891b2d9640 1 Processor -- start 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f891b2d9640 1 -- start start 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f891b2d9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8914100ad0 0x7f8914104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f891b2d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914101cd0 0x7f89141030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f891b2d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 0x7f89141035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f891b2d9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f891410c750 con 0x7f8914102690 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f891b2d9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f891410c5d0 con 0x7f8914100ad0 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f891b2d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f891410c8d0 con 0x7f8914101cd0 2024-09-19T03:58:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f8919876640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 0x7f89141035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f8918874640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914101cd0 0x7f89141030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f8919876640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 0x7f89141035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53848/0 (socket says 172.21.15.19:53848) 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f8918874640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914101cd0 0x7f89141030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37420/0 (socket says 172.21.15.19:37420) 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f8919876640 1 -- 172.21.15.19:0/3230496041 learned_addr learned my addr 172.21.15.19:0/3230496041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.740+0000 7f8919075640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8914100ad0 0x7f8914104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f8919876640 1 -- 172.21.15.19:0/3230496041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914101cd0 msgr2=0x7f89141030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f8919876640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914101cd0 0x7f89141030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f8919876640 1 -- 172.21.15.19:0/3230496041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8914100ad0 msgr2=0x7f8914104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:58:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f8919876640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8914100ad0 0x7f8914104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f8919876640 1 -- 172.21.15.19:0/3230496041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8914103eb0 con 0x7f8914102690 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f8918874640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914101cd0 0x7f89141030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f8919075640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8914100ad0 0x7f8914104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f8919876640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 0x7f89141035f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f890c0314e0 tx=0x7f890c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c03f030 con 0x7f8914102690 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89141a1fc0 con 0x7f8914102690 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c003ce0 con 0x7f8914102690 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c035070 con 0x7f8914102690 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.741+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89141a2420 con 0x7f8914102690 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.742+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f890c042030 con 0x7f8914102690 2024-09-19T03:58:40.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.742+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89141a2650 con 0x7f8914102690 2024-09-19T03:58:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.742+0000 7f890a7fc640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89000394d0 0x7f890003b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:58:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.742+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f890c06cbb0 con 0x7f8914102690 2024-09-19T03:58:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.743+0000 7f8919075640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89000394d0 0x7f890003b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:58:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.743+0000 7f8919075640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89000394d0 0x7f890003b990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8914101b30 tx=0x7f8910009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:58:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.747+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f890c04c350 con 0x7f8914102690 2024-09-19T03:58:40.905 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:40.807+0000 7fb3f9cf8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T03:58:40.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:40.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:40.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:40.933+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f89140fe490 con 0x7f89000394d0 2024-09-19T03:58:41.001 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.707+0000 7f51db687040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T03:58:41.001 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.767+0000 7f51db687040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T03:58:41.001 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.933+0000 7f51db687040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T03:58:41.001 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:40.999+0000 7f51db687040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T03:58:41.268 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:41.068+0000 7f51db687040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T03:58:41.268 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:41.149+0000 7f51db687040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T03:58:41.268 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:41.208+0000 7f51db687040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T03:58:41.268 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:41.266+0000 7f51db687040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T03:58:42.155 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:41.693+0000 7fb3f9cf8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T03:58:42.155 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:41.758+0000 7fb3f9cf8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T03:58:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T03:58:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:42.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:42.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:41 smithi111 ceph-mon[35365]: Reconfiguring daemon mgr.x on smithi019 2024-09-19T03:58:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:41 smithi019 ceph-mon[30938]: Reconfiguring daemon mgr.x on smithi019 2024-09-19T03:58:42.208 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 03:58:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z[37289]: 2024-09-19T03:58:42.054+0000 7f51db687040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T03:58:42.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:42.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T03:58:42.210 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T03:58:42.210 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T03:58:42.210 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:42.210 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:41 smithi196 ceph-mon[35731]: Reconfiguring daemon mgr.x on smithi019 2024-09-19T03:58:42.727 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:42.471+0000 7fb3f9cf8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T03:58:42.727 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:42.584+0000 7fb3f9cf8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T03:58:42.727 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:42.725+0000 7fb3f9cf8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T03:58:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:42.872+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f890c0438e0 con 0x7f8914102690 2024-09-19T03:58:42.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:42 smithi019 ceph-mon[30938]: Standby manager daemon z started 2024-09-19T03:58:42.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:42 smithi019 ceph-mon[30938]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T03:58:42.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:42 smithi019 ceph-mon[30938]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T03:58:42.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:42 smithi019 ceph-mon[30938]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T03:58:42.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:42 smithi019 ceph-mon[30938]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T03:58:43.004 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:42.786+0000 7fb3f9cf8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T03:58:43.004 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:42.934+0000 7fb3f9cf8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T03:58:43.004 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:43.002+0000 7fb3f9cf8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T03:58:43.005 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-mon[35365]: Standby manager daemon z started 2024-09-19T03:58:43.005 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-mon[35365]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T03:58:43.005 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-mon[35365]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T03:58:43.005 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-mon[35365]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T03:58:43.005 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:42 smithi111 ceph-mon[35365]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T03:58:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:42 smithi196 ceph-mon[35731]: Standby manager daemon z started 2024-09-19T03:58:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:42 smithi196 ceph-mon[35731]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T03:58:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:42 smithi196 ceph-mon[35731]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T03:58:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:42 smithi196 ceph-mon[35731]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T03:58:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:42 smithi196 ceph-mon[35731]: from='mgr.? 172.21.15.196:0/3315698225' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T03:58:43.284 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:43.062+0000 7fb3f9cf8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T03:58:43.284 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:43.218+0000 7fb3f9cf8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T03:58:43.284 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:43.283+0000 7fb3f9cf8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T03:58:43.549 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:43.348+0000 7fb3f9cf8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T03:58:43.549 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:43.430+0000 7fb3f9cf8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T03:58:43.549 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:43.488+0000 7fb3f9cf8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T03:58:43.549 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:43.547+0000 7fb3f9cf8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T03:58:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T03:58:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T03:58:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:43.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:43.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:43.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T03:58:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T03:58:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:44.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:44.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T03:58:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T03:58:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:58:44.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:44.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:58:44.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:58:44.655 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y[36729]: 2024-09-19T03:58:44.337+0000 7fb3f9cf8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T03:58:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:58:44.881+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f890c039350 con 0x7f8914102690 2024-09-19T03:58:45.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: Standby manager daemon y started 2024-09-19T03:58:45.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T03:58:45.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T03:58:45.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T03:58:45.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T03:58:45.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1894577488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab98d168-5199-4c3f-a220-00c832addff2"}]: dispatch 2024-09-19T03:58:45.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1894577488' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab98d168-5199-4c3f-a220-00c832addff2"}]': finished 2024-09-19T03:58:45.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T03:58:45.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:58:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: Standby manager daemon y started 2024-09-19T03:58:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T03:58:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T03:58:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T03:58:45.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T03:58:45.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1894577488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab98d168-5199-4c3f-a220-00c832addff2"}]: dispatch 2024-09-19T03:58:45.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1894577488' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab98d168-5199-4c3f-a220-00c832addff2"}]': finished 2024-09-19T03:58:45.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T03:58:45.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:58:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: Standby manager daemon y started 2024-09-19T03:58:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T03:58:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T03:58:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T03:58:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: from='mgr.? 172.21.15.111:0/1777027014' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T03:58:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1894577488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab98d168-5199-4c3f-a220-00c832addff2"}]: dispatch 2024-09-19T03:58:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1894577488' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab98d168-5199-4c3f-a220-00c832addff2"}]': finished 2024-09-19T03:58:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T03:58:45.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:58:46.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:45 smithi111 ceph-mon[35365]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:46.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:45 smithi111 ceph-mon[35365]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T03:58:46.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:45 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T03:58:46.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:45 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2148890825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:58:46.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:45 smithi019 ceph-mon[30938]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:46.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:45 smithi019 ceph-mon[30938]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T03:58:46.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:45 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T03:58:46.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:45 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2148890825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:58:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:45 smithi196 ceph-mon[35731]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:45 smithi196 ceph-mon[35731]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T03:58:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:45 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T03:58:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:45 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2148890825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:58:48.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:47 smithi111 ceph-mon[35365]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:48.188 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:47 smithi019 ceph-mon[30938]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:48.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:47 smithi196 ceph-mon[35731]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:50.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:49 smithi111 ceph-mon[35365]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:50.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:49 smithi019 ceph-mon[30938]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:50.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:49 smithi196 ceph-mon[35731]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:51 smithi019 ceph-mon[30938]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:52.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:51 smithi196 ceph-mon[35731]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:51 smithi111 ceph-mon[35365]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:53.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T03:58:53.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:53.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T03:58:53.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:53.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T03:58:53.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:58:54.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:53 smithi019 ceph-mon[30938]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:53 smithi019 ceph-mon[30938]: Deploying daemon osd.0 on smithi019 2024-09-19T03:58:54.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:53 smithi196 ceph-mon[35731]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:54.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:53 smithi196 ceph-mon[35731]: Deploying daemon osd.0 on smithi019 2024-09-19T03:58:54.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:53 smithi111 ceph-mon[35365]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:54.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:53 smithi111 ceph-mon[35365]: Deploying daemon osd.0 on smithi019 2024-09-19T03:58:56.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:55 smithi019 ceph-mon[30938]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:56.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:55 smithi196 ceph-mon[35731]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:56.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:55 smithi111 ceph-mon[35365]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:58.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:57 smithi019 ceph-mon[30938]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:58.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:57 smithi196 ceph-mon[35731]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:58:58.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:57 smithi111 ceph-mon[35365]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:00.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:59 smithi196 ceph-mon[35731]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:00.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:00.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:00.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:58:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:59 smithi111 ceph-mon[35365]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:58:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:59 smithi019 ceph-mon[30938]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:58:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:01 smithi019 ceph-mon[30938]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:02.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:02.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:02.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:01 smithi196 ceph-mon[35731]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:02.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:02.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:02.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:01 smithi111 ceph-mon[35365]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-09-19T03:59:03.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.340+0000 7f890a7fc640 1 -- 172.21.15.19:0/3230496041 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89140fe490 con 0x7f89000394d0 2024-09-19T03:59:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89000394d0 msgr2=0x7f890003b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89000394d0 0x7f890003b990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8914101b30 tx=0x7f8910009500 comp rx=0 tx=0).stop 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 msgr2=0x7f89141035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 0x7f89141035f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f890c0314e0 tx=0x7f890c0042f0 comp rx=0 tx=0).stop 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 shutdown_connections 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89000394d0 0x7f890003b990 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8914102690 0x7f89141035f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914101cd0 0x7f89141030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 --2- 172.21.15.19:0/3230496041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8914100ad0 0x7f8914104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 >> 172.21.15.19:0/3230496041 conn(0x7f89140fc280 msgr2=0x7f89140fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 shutdown_connections 2024-09-19T03:59:03.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:03.343+0000 7f891b2d9640 1 -- 172.21.15.19:0/3230496041 wait complete. 2024-09-19T03:59:03.962 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T03:59:03.965 INFO:tasks.cephadm:Deploying osd.1 on smithi111 with /dev/vg_nvme/lv_4... 2024-09-19T03:59:03.965 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T03:59:04.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:03 smithi019 ceph-mon[30938]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:04.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:04.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:04.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:04.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:03 smithi196 ceph-mon[35731]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:04.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:03 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:04.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:03 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:04.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:03 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T03:59:04.337 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:03 smithi111 ceph-mon[35365]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:04.337 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:03 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:04.337 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:03 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:04.337 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:03 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:05.181 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 03:59:05 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0[44515]: 2024-09-19T03:59:05.145+0000 7f7360e47640 -1 osd.0 0 log_to_monitors true 2024-09-19T03:59:06.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:05 smithi196 ceph-mon[35731]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:06.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:05 smithi196 ceph-mon[35731]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T03:59:06.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:05 smithi019 ceph-mon[30938]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:06.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:05 smithi019 ceph-mon[30938]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T03:59:06.342 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:05 smithi111 ceph-mon[35365]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:06.342 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:05 smithi111 ceph-mon[35365]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T03:59:06.647 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T03:59:06.673 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2024-09-19T03:59:06.918 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T03:59:07.213 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:06 smithi111 ceph-mon[35365]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T03:59:07.214 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:06 smithi111 ceph-mon[35365]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T03:59:07.214 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:06 smithi111 ceph-mon[35365]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T03:59:07.214 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:06 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:07.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 -- 172.21.15.111:0/2099698873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 msgr2=0x7f5e68100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:07.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 --2- 172.21.15.111:0/2099698873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 0x7f5e68100ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e5802a410 comp rx=0 tx=0).stop 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 -- 172.21.15.111:0/2099698873 shutdown_connections 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 --2- 172.21.15.111:0/2099698873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68102690 0x7f5e68108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 --2- 172.21.15.111:0/2099698873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e68101cd0 0x7f5e68102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 --2- 172.21.15.111:0/2099698873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 0x7f5e68100ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 -- 172.21.15.111:0/2099698873 >> 172.21.15.111:0/2099698873 conn(0x7f5e680fc280 msgr2=0x7f5e680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 -- 172.21.15.111:0/2099698873 shutdown_connections 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.413+0000 7f5e707ae640 1 -- 172.21.15.111:0/2099698873 wait complete. 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e707ae640 1 Processor -- start 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e707ae640 1 -- start start 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e707ae640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 0x7f5e6819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e707ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68101cd0 0x7f5e6819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:07.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e707ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e68102690 0x7f5e681a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e707ae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810c750 con 0x7f5e68101cd0 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e707ae640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810c5d0 con 0x7f5e68100ad0 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e707ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810c8d0 con 0x7f5e68102690 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6e54a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 0x7f5e6819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6e54a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 0x7f5e6819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60352/0 (socket says 172.21.15.111:60352) 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6e54a640 1 -- 172.21.15.111:0/4268444636 learned_addr learned my addr 172.21.15.111:0/4268444636 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6dd49640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68101cd0 0x7f5e6819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6ed4b640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e68102690 0x7f5e681a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6e54a640 1 -- 172.21.15.111:0/4268444636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e68102690 msgr2=0x7f5e681a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6e54a640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e68102690 0x7f5e681a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6e54a640 1 -- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68101cd0 msgr2=0x7f5e6819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:07.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6e54a640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68101cd0 0x7f5e6819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6e54a640 1 -- 172.21.15.111:0/4268444636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e681a1e60 con 0x7f5e68100ad0 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6dd49640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68101cd0 0x7f5e6819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.414+0000 7f5e6ed4b640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e68102690 0x7f5e681a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.415+0000 7f5e6e54a640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 0x7f5e6819a230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006010 tx=0x7f5e580043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.415+0000 7f5e5f7fe640 1 -- 172.21.15.111:0/4268444636 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5803f030 con 0x7f5e68100ad0 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.415+0000 7f5e5f7fe640 1 -- 172.21.15.111:0/4268444636 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58003ce0 con 0x7f5e68100ad0 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.415+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e681a20f0 con 0x7f5e68100ad0 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.415+0000 7f5e5f7fe640 1 -- 172.21.15.111:0/4268444636 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58035070 con 0x7f5e68100ad0 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.415+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681a25d0 con 0x7f5e68100ad0 2024-09-19T03:59:07.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.416+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e681a2830 con 0x7f5e68100ad0 2024-09-19T03:59:07.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.417+0000 7f5e5f7fe640 1 -- 172.21.15.111:0/4268444636 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f5e58042030 con 0x7f5e68100ad0 2024-09-19T03:59:07.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.418+0000 7f5e5f7fe640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e540a31e0 0x7f5e540a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:07.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.418+0000 7f5e5f7fe640 1 -- 172.21.15.111:0/4268444636 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f5e580ee5e0 con 0x7f5e68100ad0 2024-09-19T03:59:07.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.418+0000 7f5e6dd49640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e540a31e0 0x7f5e540a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:07.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.419+0000 7f5e6dd49640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e540a31e0 0x7f5e540a56a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e6819b750 tx=0x7f5e64005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:59:07.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.421+0000 7f5e5f7fe640 1 -- 172.21.15.111:0/4268444636 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5e580bce90 con 0x7f5e68100ad0 2024-09-19T03:59:07.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:06 smithi019 ceph-mon[30938]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T03:59:07.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:06 smithi019 ceph-mon[30938]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T03:59:07.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:06 smithi019 ceph-mon[30938]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T03:59:07.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:06 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:07.432 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 03:59:06 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0[44515]: 2024-09-19T03:59:06.961+0000 7f735cdca640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T03:59:07.432 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 03:59:06 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0[44515]: 2024-09-19T03:59:06.964+0000 7f7358bf2640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T03:59:07.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:06 smithi196 ceph-mon[35731]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T03:59:07.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:06 smithi196 ceph-mon[35731]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T03:59:07.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:06 smithi196 ceph-mon[35731]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T03:59:07.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:06 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:07.598 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:07.596+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5e6806a710 con 0x7f5e540a31e0 2024-09-19T03:59:08.300 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:08.300 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi019 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:08.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:08.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:08.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:08.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:07 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:08.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi019 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-19T03:59:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:08.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:08.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:08.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:08.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:07 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T03:59:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T03:59:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi019 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:08.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:07 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:09.202 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:08 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T03:59:09.203 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:08 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T03:59:09.203 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:08 smithi111 ceph-mon[35365]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:09.203 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:08 smithi111 ceph-mon[35365]: osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] boot 2024-09-19T03:59:09.203 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:08 smithi111 ceph-mon[35365]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T03:59:09.203 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:08 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:09.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:08 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T03:59:09.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:08 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T03:59:09.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:08 smithi019 ceph-mon[30938]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:09.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:08 smithi019 ceph-mon[30938]: osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] boot 2024-09-19T03:59:09.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:08 smithi019 ceph-mon[30938]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T03:59:09.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:08 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:09.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:08 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T03:59:09.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:08 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T03:59:09.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:08 smithi196 ceph-mon[35731]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:09.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:08 smithi196 ceph-mon[35731]: osd.0 [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] boot 2024-09-19T03:59:09.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:08 smithi196 ceph-mon[35731]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T03:59:09.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:08 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T03:59:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:09 smithi111 ceph-mon[35365]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:09 smithi111 ceph-mon[35365]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T03:59:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:09 smithi111 ceph-mon[35365]: from='client.? 172.21.15.111:0/2597234552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]: dispatch 2024-09-19T03:59:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:09 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]: dispatch 2024-09-19T03:59:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:09 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]': finished 2024-09-19T03:59:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:09 smithi111 ceph-mon[35365]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T03:59:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:09 smithi019 ceph-mon[30938]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:09 smithi019 ceph-mon[30938]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T03:59:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:09 smithi019 ceph-mon[30938]: from='client.? 172.21.15.111:0/2597234552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]: dispatch 2024-09-19T03:59:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:09 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]: dispatch 2024-09-19T03:59:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:09 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]': finished 2024-09-19T03:59:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:09 smithi019 ceph-mon[30938]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T03:59:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:09 smithi196 ceph-mon[35731]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T03:59:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:09 smithi196 ceph-mon[35731]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T03:59:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:09 smithi196 ceph-mon[35731]: from='client.? 172.21.15.111:0/2597234552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]: dispatch 2024-09-19T03:59:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:09 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]: dispatch 2024-09-19T03:59:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:09 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66cb950f-6aee-4140-8411-0e9deaf594fc"}]': finished 2024-09-19T03:59:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:09 smithi196 ceph-mon[35731]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T03:59:10.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:11.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:10 smithi111 ceph-mon[35365]: from='client.? 172.21.15.111:0/698524213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:59:11.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:10 smithi019 ceph-mon[30938]: from='client.? 172.21.15.111:0/698524213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:59:11.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:10 smithi196 ceph-mon[35731]: from='client.? 172.21.15.111:0/698524213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:59:12.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:11 smithi111 ceph-mon[35365]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:12.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:11 smithi019 ceph-mon[30938]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:12.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:11 smithi196 ceph-mon[35731]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:14.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:13 smithi019 ceph-mon[30938]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:13 smithi111 ceph-mon[35365]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:13 smithi196 ceph-mon[35731]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:16.139 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:15 smithi111 ceph-mon[35365]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:15 smithi019 ceph-mon[30938]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:16.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:15 smithi196 ceph-mon[35731]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:18.390 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:17 smithi111 ceph-mon[35365]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:18.390 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T03:59:18.390 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:18.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:17 smithi019 ceph-mon[30938]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:18.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T03:59:18.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:17 smithi196 ceph-mon[35731]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T03:59:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:19.354 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:18 smithi111 ceph-mon[35365]: Deploying daemon osd.1 on smithi111 2024-09-19T03:59:19.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:18 smithi019 ceph-mon[30938]: Deploying daemon osd.1 on smithi111 2024-09-19T03:59:19.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:18 smithi196 ceph-mon[35731]: Deploying daemon osd.1 on smithi111 2024-09-19T03:59:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:19 smithi111 ceph-mon[35365]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:20.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:19 smithi196 ceph-mon[35731]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:20.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:19 smithi019 ceph-mon[30938]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:22.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:21 smithi111 ceph-mon[35365]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:22.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:21 smithi019 ceph-mon[30938]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:22.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:21 smithi196 ceph-mon[35731]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:24.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:23 smithi111 ceph-mon[35365]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:24.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:23 smithi019 ceph-mon[30938]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:24.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:23 smithi196 ceph-mon[35731]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:25.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:25.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:25.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:25.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:25.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:25.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:26.576 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:26 smithi111 ceph-mon[35365]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:26.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:26 smithi019 ceph-mon[30938]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:26.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:26 smithi196 ceph-mon[35731]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:27.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:27 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:27 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:27 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:27.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:27 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:27.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:27 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:27 smithi111 ceph-mon[35365]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:27.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:27 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:27 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:27 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:27.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:27 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:27.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:27 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:27 smithi019 ceph-mon[30938]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:27.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:27 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:27 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:27 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:27.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:27 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:27.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:27 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:27.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:27 smithi196 ceph-mon[35731]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:28.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.475+0000 7f5e5f7fe640 1 -- 172.21.15.111:0/4268444636 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5e6806a710 con 0x7f5e540a31e0 2024-09-19T03:59:28.477 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 1 on host 'smithi111' 2024-09-19T03:59:28.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e540a31e0 msgr2=0x7f5e540a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:28.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e540a31e0 0x7f5e540a56a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e6819b750 tx=0x7f5e64005e50 comp rx=0 tx=0).stop 2024-09-19T03:59:28.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 msgr2=0x7f5e6819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:28.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 0x7f5e6819a230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006010 tx=0x7f5e580043b0 comp rx=0 tx=0).stop 2024-09-19T03:59:28.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 shutdown_connections 2024-09-19T03:59:28.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e540a31e0 0x7f5e540a56a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:28.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e68102690 0x7f5e681a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:28.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68101cd0 0x7f5e6819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:28.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 --2- 172.21.15.111:0/4268444636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68100ad0 0x7f5e6819a230 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:28.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.477+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 >> 172.21.15.111:0/4268444636 conn(0x7f5e680fc280 msgr2=0x7f5e6806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:59:28.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.478+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 shutdown_connections 2024-09-19T03:59:28.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T03:59:28.478+0000 7f5e707ae640 1 -- 172.21.15.111:0/4268444636 wait complete. 2024-09-19T03:59:29.125 DEBUG:teuthology.orchestra.run.smithi111:osd.1> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T03:59:29.127 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-09-19T03:59:29.127 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T03:59:29.378 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T03:59:29.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:29 smithi196 ceph-mon[35731]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:29.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:29 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:29.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:29 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:29.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:29 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:29.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:29 smithi111 ceph-mon[35365]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:29.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:29 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:29.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:29 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:29.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:29 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:29.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:29 smithi019 ceph-mon[30938]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:29.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:29 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:29.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:29 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:29.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:29 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:30.115 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 03:59:29 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1[39818]: 2024-09-19T03:59:29.825+0000 7fb1370f3640 -1 osd.1 0 log_to_monitors true 2024-09-19T03:59:30.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:30 smithi111 ceph-mon[35365]: from='osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T03:59:30.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:30 smithi111 ceph-mon[35365]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T03:59:30.669 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:30 smithi196 ceph-mon[35731]: from='osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T03:59:30.669 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:30 smithi196 ceph-mon[35731]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T03:59:30.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:30 smithi019 ceph-mon[30938]: from='osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T03:59:30.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:30 smithi019 ceph-mon[30938]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T03:59:31.607 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T03:59:31.634 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-09-19T03:59:31.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:31 smithi019 ceph-mon[30938]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:31.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:31 smithi019 ceph-mon[30938]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T03:59:31.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:31 smithi019 ceph-mon[30938]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T03:59:31.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:31.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:31 smithi019 ceph-mon[30938]: from='osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T03:59:31.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:31 smithi019 ceph-mon[30938]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T03:59:31.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:31 smithi196 ceph-mon[35731]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:31.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:31 smithi196 ceph-mon[35731]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T03:59:31.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:31 smithi196 ceph-mon[35731]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T03:59:31.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:31.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:31 smithi196 ceph-mon[35731]: from='osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T03:59:31.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:31 smithi196 ceph-mon[35731]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T03:59:31.722 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 03:59:31 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1[39818]: 2024-09-19T03:59:31.419+0000 7fb133076640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T03:59:31.722 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 03:59:31 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1[39818]: 2024-09-19T03:59:31.422+0000 7fb12ee9e640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T03:59:31.722 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:31 smithi111 ceph-mon[35365]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:31.722 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:31 smithi111 ceph-mon[35365]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T03:59:31.722 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:31 smithi111 ceph-mon[35365]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T03:59:31.722 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:31.722 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:31 smithi111 ceph-mon[35365]: from='osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T03:59:31.722 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:31 smithi111 ceph-mon[35365]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T03:59:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T03:59:32.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 -- 172.21.15.196:0/3046302531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 msgr2=0x7fe058102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:32.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 --2- 172.21.15.196:0/3046302531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 0x7fe058102150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe048009900 tx=0x7fe04802a410 comp rx=0 tx=0).stop 2024-09-19T03:59:32.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 -- 172.21.15.196:0/3046302531 shutdown_connections 2024-09-19T03:59:32.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 --2- 172.21.15.196:0/3046302531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe058102690 0x7fe058108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:32.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 --2- 172.21.15.196:0/3046302531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 0x7fe058102150 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:32.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 --2- 172.21.15.196:0/3046302531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058100ad0 0x7fe058100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:32.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 -- 172.21.15.196:0/3046302531 >> 172.21.15.196:0/3046302531 conn(0x7fe0580fc280 msgr2=0x7fe0580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:59:32.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 -- 172.21.15.196:0/3046302531 shutdown_connections 2024-09-19T03:59:32.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 -- 172.21.15.196:0/3046302531 wait complete. 2024-09-19T03:59:32.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.308+0000 7fe05fc21640 1 Processor -- start 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05fc21640 1 -- start start 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05fc21640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe058100ad0 0x7fe05806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05fc21640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 0x7fe05819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05fc21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058102690 0x7fe05819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05fc21640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810c7a0 con 0x7fe058102690 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05fc21640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810c620 con 0x7fe058100ad0 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05fc21640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810c920 con 0x7fe058101cd0 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 0x7fe05819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 0x7fe05819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:50978/0 (socket says 172.21.15.196:50978) 2024-09-19T03:59:32.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 -- 172.21.15.196:0/1994130882 learned_addr learned my addr 172.21.15.196:0/1994130882 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d9bd640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe058100ad0 0x7fe05806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05e1be640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058102690 0x7fe05819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 -- 172.21.15.196:0/1994130882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe058100ad0 msgr2=0x7fe05806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe058100ad0 0x7fe05806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 -- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058102690 msgr2=0x7fe05819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058102690 0x7fe05819f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 -- 172.21.15.196:0/1994130882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05819fb50 con 0x7fe058101cd0 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.309+0000 7fe05d1bc640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 0x7fe05819cc20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe048031440 tx=0x7fe048032e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.310+0000 7fe04effd640 1 -- 172.21.15.196:0/1994130882 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04803f030 con 0x7fe058101cd0 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.310+0000 7fe04effd640 1 -- 172.21.15.196:0/1994130882 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe048003ce0 con 0x7fe058101cd0 2024-09-19T03:59:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.310+0000 7fe04effd640 1 -- 172.21.15.196:0/1994130882 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048035070 con 0x7fe058101cd0 2024-09-19T03:59:32.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.310+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05819fd80 con 0x7fe058101cd0 2024-09-19T03:59:32.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.310+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581a01b0 con 0x7fe058101cd0 2024-09-19T03:59:32.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.311+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe058107ed0 con 0x7fe058101cd0 2024-09-19T03:59:32.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.312+0000 7fe04effd640 1 -- 172.21.15.196:0/1994130882 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fe048042030 con 0x7fe058101cd0 2024-09-19T03:59:32.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.312+0000 7fe04effd640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe0440a3230 0x7fe0440a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:32.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.313+0000 7fe04effd640 1 -- 172.21.15.196:0/1994130882 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fe0480ee540 con 0x7fe058101cd0 2024-09-19T03:59:32.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.313+0000 7fe05d9bd640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe0440a3230 0x7fe0440a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:32.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.313+0000 7fe05d9bd640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe0440a3230 0x7fe0440a56f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe058101b30 tx=0x7fe05400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:59:32.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.317+0000 7fe04effd640 1 -- 172.21.15.196:0/1994130882 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe0480bcbd0 con 0x7fe058101cd0 2024-09-19T03:59:32.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:32.496+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe058062630 con 0x7fe0440a3230 2024-09-19T03:59:32.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T03:59:32.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T03:59:32.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi111 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:32.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T03:59:32.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T03:59:32.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:32.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:32.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi111 2024-09-19T03:59:32.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:32.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-19T03:59:32.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:32.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:32.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T03:59:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T03:59:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi111 2024-09-19T03:59:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:32.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-19T03:59:32.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:32.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:32.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:32.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:33.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T03:59:33.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T03:59:33.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:33.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] boot 2024-09-19T03:59:33.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T03:59:33.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:33.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:33.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:33.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:33.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:33.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T03:59:33.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T03:59:33.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:33.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] boot 2024-09-19T03:59:33.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T03:59:33.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:33.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:33.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:33.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:33.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T03:59:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T03:59:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T03:59:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: osd.1 [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] boot 2024-09-19T03:59:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T03:59:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T03:59:33.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:33.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:33.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:33.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:34.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:34 smithi196 ceph-mon[35731]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T03:59:34.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:34 smithi019 ceph-mon[30938]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T03:59:34.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:34 smithi111 ceph-mon[35365]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T03:59:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:35 smithi196 ceph-mon[35731]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:35 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/1976078526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]: dispatch 2024-09-19T03:59:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:35 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]: dispatch 2024-09-19T03:59:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:35 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]': finished 2024-09-19T03:59:35.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:35 smithi196 ceph-mon[35731]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T03:59:35.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:35.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:35 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/1801500204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:59:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:35 smithi111 ceph-mon[35365]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:35 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/1976078526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]: dispatch 2024-09-19T03:59:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:35 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]: dispatch 2024-09-19T03:59:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:35 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]': finished 2024-09-19T03:59:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:35 smithi111 ceph-mon[35365]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T03:59:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:35 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/1801500204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:59:35.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:35 smithi019 ceph-mon[30938]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:35.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:35 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/1976078526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]: dispatch 2024-09-19T03:59:35.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:35 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]: dispatch 2024-09-19T03:59:35.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:35 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2ad4ffa-1018-4519-96f8-a3ef93a779bb"}]': finished 2024-09-19T03:59:35.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:35 smithi019 ceph-mon[30938]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T03:59:35.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:35.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:35 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/1801500204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T03:59:37.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:37 smithi196 ceph-mon[35731]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:37.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:37 smithi111 ceph-mon[35365]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:37.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:37 smithi019 ceph-mon[30938]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:39.662 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:39 smithi196 ceph-mon[35731]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:39.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:39 smithi111 ceph-mon[35365]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:39.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:39 smithi019 ceph-mon[30938]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:41 smithi196 ceph-mon[35731]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:41.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:41 smithi111 ceph-mon[35365]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:41.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:41 smithi019 ceph-mon[30938]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:43.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:43 smithi196 ceph-mon[35731]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:43.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T03:59:43.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:43.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:43 smithi196 ceph-mon[35731]: Deploying daemon osd.2 on smithi196 2024-09-19T03:59:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:43 smithi111 ceph-mon[35365]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T03:59:43.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:43.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:43 smithi111 ceph-mon[35365]: Deploying daemon osd.2 on smithi196 2024-09-19T03:59:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:43 smithi019 ceph-mon[30938]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T03:59:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:43.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:43 smithi019 ceph-mon[30938]: Deploying daemon osd.2 on smithi196 2024-09-19T03:59:45.668 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:45 smithi196 ceph-mon[35731]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:45.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:45 smithi111 ceph-mon[35365]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:45.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:45 smithi019 ceph-mon[30938]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:47.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:47 smithi196 ceph-mon[35731]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:47.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:47 smithi111 ceph-mon[35365]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:47.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:47 smithi019 ceph-mon[30938]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:49.707 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:49 smithi111 ceph-mon[35365]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:49.707 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:49.707 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:49.708 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:49 smithi196 ceph-mon[35731]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:49.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:49 smithi019 ceph-mon[30938]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:49.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:49.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:49.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.404 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:52 smithi196 ceph-mon[35731]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:52.404 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.405 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.405 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:52.405 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:52.405 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.447+0000 7fe04effd640 1 -- 172.21.15.196:0/1994130882 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe058062630 con 0x7fe0440a3230 2024-09-19T03:59:52.449 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-09-19T03:59:52.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe0440a3230 msgr2=0x7fe0440a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:52.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe0440a3230 0x7fe0440a56f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe058101b30 tx=0x7fe05400a2d0 comp rx=0 tx=0).stop 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 msgr2=0x7fe05819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 0x7fe05819cc20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe048031440 tx=0x7fe048032e20 comp rx=0 tx=0).stop 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 shutdown_connections 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe0440a3230 0x7fe0440a56f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058102690 0x7fe05819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe058101cd0 0x7fe05819cc20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 --2- 172.21.15.196:0/1994130882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe058100ad0 0x7fe05806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 >> 172.21.15.196:0/1994130882 conn(0x7fe0580fc280 msgr2=0x7fe0581050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 shutdown_connections 2024-09-19T03:59:52.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T03:59:52.450+0000 7fe05fc21640 1 -- 172.21.15.196:0/1994130882 wait complete. 2024-09-19T03:59:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:52 smithi111 ceph-mon[35365]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:52.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:52 smithi019 ceph-mon[30938]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:53.043 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T03:59:53.046 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-09-19T03:59:53.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T03:59:53.319 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:59:53.377 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:53 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:53.377 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:53 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:53.378 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:53 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:53.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:53.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:53.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:53 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:53.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:53 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T03:59:53.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:53 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:53.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:53 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:54.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:54 smithi196 ceph-mon[35731]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:54.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:54 smithi019 ceph-mon[30938]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:54.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:54 smithi111 ceph-mon[35365]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:54.708 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 03:59:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2[40533]: 2024-09-19T03:59:54.562+0000 7f414529f640 -1 osd.2 0 log_to_monitors true 2024-09-19T03:59:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:55 smithi196 ceph-mon[35731]: from='osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T03:59:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:55 smithi196 ceph-mon[35731]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T03:59:55.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:55 smithi019 ceph-mon[30938]: from='osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T03:59:55.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:55 smithi019 ceph-mon[30938]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T03:59:55.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:55 smithi111 ceph-mon[35365]: from='osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T03:59:55.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:55 smithi111 ceph-mon[35365]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T03:59:55.936 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T03:59:55.964 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-09-19T03:59:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T03:59:56.502 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:56.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:56.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:56.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:56.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:56.657 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:56.657 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.708 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2[40533]: 2024-09-19T03:59:56.243+0000 7f4141222640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T03:59:56.708 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2[40533]: 2024-09-19T03:59:56.248+0000 7f413d04a640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T03:59:56.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:56.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T03:59:56.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T03:59:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 -- 172.21.15.19:0/3848219009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348101d10 msgr2=0x7f8348102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 --2- 172.21.15.19:0/3848219009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348101d10 0x7f8348102190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f833c009900 tx=0x7f833c02a410 comp rx=0 tx=0).stop 2024-09-19T03:59:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 -- 172.21.15.19:0/3848219009 shutdown_connections 2024-09-19T03:59:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 --2- 172.21.15.19:0/3848219009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83481026d0 0x7f8348108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 --2- 172.21.15.19:0/3848219009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348101d10 0x7f8348102190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 --2- 172.21.15.19:0/3848219009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8348100b10 0x7f8348100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 -- 172.21.15.19:0/3848219009 >> 172.21.15.19:0/3848219009 conn(0x7f83480fc2a0 msgr2=0x7f83480fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 -- 172.21.15.19:0/3848219009 shutdown_connections 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 -- 172.21.15.19:0/3848219009 wait complete. 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 Processor -- start 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.688+0000 7f834dbbb640 1 -- start start 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f834dbbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348100b10 0x7f834806b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f834dbbb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8348101d10 0x7f834819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f834dbbb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83481026d0 0x7f834819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f834dbbb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f834810c790 con 0x7f8348100b10 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f834dbbb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f834810c610 con 0x7f8348101d10 2024-09-19T03:59:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f834dbbb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f834810c910 con 0x7f83481026d0 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348100b10 0x7f834806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348100b10 0x7f834806b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53676/0 (socket says 172.21.15.19:53676) 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 -- 172.21.15.19:0/177835959 learned_addr learned my addr 172.21.15.19:0/177835959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f8346ffd640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8348101d10 0x7f834819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f8347fff640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83481026d0 0x7f834819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 -- 172.21.15.19:0/177835959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83481026d0 msgr2=0x7f834819f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83481026d0 0x7f834819f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 -- 172.21.15.19:0/177835959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8348101d10 msgr2=0x7f834819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8348101d10 0x7f834819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 -- 172.21.15.19:0/177835959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834819fb40 con 0x7f8348100b10 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f83477fe640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348100b10 0x7f834806b330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f833800d770 tx=0x7f833800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f8344ff9640 1 -- 172.21.15.19:0/177835959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338017070 con 0x7f8348100b10 2024-09-19T03:59:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f8344ff9640 1 -- 172.21.15.19:0/177835959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8338009c30 con 0x7f8348100b10 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834819fe30 con 0x7f8348100b10 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f8344ff9640 1 -- 172.21.15.19:0/177835959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338003dd0 con 0x7f8348100b10 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.689+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481a0230 con 0x7f8348100b10 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.691+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8348107f10 con 0x7f8348100b10 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.692+0000 7f8344ff9640 1 -- 172.21.15.19:0/177835959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f8338003fb0 con 0x7f8348100b10 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.693+0000 7f8344ff9640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f83300a31b0 0x7f83300a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.693+0000 7f8346ffd640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f83300a31b0 0x7f83300a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.693+0000 7f8344ff9640 1 -- 172.21.15.19:0/177835959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f83380cdbc0 con 0x7f8348100b10 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.693+0000 7f8346ffd640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f83300a31b0 0x7f83300a5670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f833c000c00 tx=0x7f833c0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T03:59:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.697+0000 7f8344ff9640 1 -- 172.21.15.19:0/177835959 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f833809c030 con 0x7f8348100b10 2024-09-19T03:59:56.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T03:59:56.881+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8348062630 con 0x7f83300a31b0 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi196 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:57.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:57.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi196 2024-09-19T03:59:57.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-19T03:59:57.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T03:59:57.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T03:59:57.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:57.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:57.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:57.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:57.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:57.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi196 2024-09-19T03:59:57.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-19T03:59:57.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T03:59:57.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T03:59:57.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:57.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:57.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T03:59:57.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T03:59:57.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T03:59:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:58 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T03:59:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:58 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T03:59:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:58 smithi019 ceph-mon[30938]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:58 smithi019 ceph-mon[30938]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:58 smithi019 ceph-mon[30938]: osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] boot 2024-09-19T03:59:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:58 smithi019 ceph-mon[30938]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T03:59:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:58 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:58 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T03:59:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:58 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T03:59:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:58 smithi111 ceph-mon[35365]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:58 smithi111 ceph-mon[35365]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:58 smithi111 ceph-mon[35365]: osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] boot 2024-09-19T03:59:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:58 smithi111 ceph-mon[35365]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T03:59:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:58 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:58.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:58 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T03:59:58.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:58 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T03:59:58.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:58 smithi196 ceph-mon[35731]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T03:59:58.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:58 smithi196 ceph-mon[35731]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T03:59:58.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:58 smithi196 ceph-mon[35731]: osd.2 [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] boot 2024-09-19T03:59:58.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:58 smithi196 ceph-mon[35731]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T03:59:58.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:58 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T03:59:59.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:59 smithi019 ceph-mon[30938]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T03:59:59.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 03:59:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T03:59:59.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:59 smithi111 ceph-mon[35365]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T03:59:59.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 03:59:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T03:59:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:59 smithi196 ceph-mon[35731]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T03:59:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 03:59:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:00:00.181 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 03:59:59 smithi019 sudo[49393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:00:00.181 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 03:59:59 smithi019 sudo[49393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:00.181 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 03:59:59 smithi019 sudo[49393]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:00.181 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 03:59:59 smithi019 sudo[49393]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:00.634 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:00.634 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:00:00.634 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:00:00.634 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:00:00.634 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1679384181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c13f30d-2850-46f4-a0d5-9f8d96b699ce"}]: dispatch 2024-09-19T04:00:00.634 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:00:00.635 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1679384181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c13f30d-2850-46f4-a0d5-9f8d96b699ce"}]': finished 2024-09-19T04:00:00.635 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:00:00.635 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:00.635 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/675340708' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:00.635 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:00 smithi111 ceph-mon[35365]: overall HEALTH_OK 2024-09-19T04:00:00.681 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:00:00 smithi019 sudo[49443]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:00:00.681 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:00:00 smithi019 sudo[49443]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:00.681 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:00:00 smithi019 sudo[49443]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:00.681 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:00:00 smithi019 sudo[49443]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1679384181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c13f30d-2850-46f4-a0d5-9f8d96b699ce"}]: dispatch 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1679384181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c13f30d-2850-46f4-a0d5-9f8d96b699ce"}]': finished 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/675340708' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:00.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:00 smithi019 ceph-mon[30938]: overall HEALTH_OK 2024-09-19T04:00:00.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:00.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1679384181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c13f30d-2850-46f4-a0d5-9f8d96b699ce"}]: dispatch 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1679384181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c13f30d-2850-46f4-a0d5-9f8d96b699ce"}]': finished 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/675340708' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:00.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:00 smithi196 ceph-mon[35731]: overall HEALTH_OK 2024-09-19T04:00:00.888 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:00:00 smithi111 sudo[43968]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:00:00.888 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:00:00 smithi111 sudo[43968]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:00.888 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:00:00 smithi111 sudo[43968]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:00.888 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:00:00 smithi111 sudo[43968]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:01.155 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:00:00 smithi111 sudo[43971]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:00:01.155 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:00:00 smithi111 sudo[43971]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:01.155 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:00:00 smithi111 sudo[43971]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:01.155 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:00:00 smithi111 sudo[43971]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:01.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:01.365+0000 7f8344ff9640 1 -- 172.21.15.19:0/177835959 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8338097fa0 con 0x7f8348100b10 2024-09-19T04:00:01.417 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:00:01 smithi196 sudo[44157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:00:01.417 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:00:01 smithi196 sudo[44157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:01.417 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:00:01 smithi196 sudo[44157]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:01.417 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:00:01 smithi196 sudo[44157]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:01.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:01 smithi196 ceph-mon[35731]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:01.418 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:01 smithi196 ceph-mon[35731]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:00:01.418 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:01.418 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:01.418 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:01.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:01 smithi111 ceph-mon[35365]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:01.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:01 smithi111 ceph-mon[35365]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:00:01.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:01.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:01.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:01.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 ceph-mon[30938]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:01.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 ceph-mon[30938]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:00:01.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:01.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:01.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:01.708 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:00:01 smithi196 sudo[44160]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:00:01.708 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:00:01 smithi196 sudo[44160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:01.708 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:00:01 smithi196 sudo[44160]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:01.708 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:00:01 smithi196 sudo[44160]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:02.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 sudo[49818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:00:02.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 sudo[49818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:02.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 sudo[49818]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:02.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:01 smithi019 sudo[49818]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 sudo[49826]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:00:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 sudo[49826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 sudo[49826]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 sudo[49826]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:02.840 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 sudo[44166]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:00:02.840 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 sudo[44166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:02.840 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 sudo[44166]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:02.840 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 sudo[44166]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:03.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:00:03.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:03.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:03.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:03.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:03.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:03.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:03.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:03.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:03.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:02 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:03.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 sudo[43974]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:00:03.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 sudo[43974]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:03.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 sudo[43974]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:03.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 sudo[43974]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:03.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:00:03.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:03.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:03.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:03.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:03.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:03.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:03.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:03.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:03.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:02 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:03.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 sudo[44169]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:00:03.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 sudo[44169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 sudo[44169]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 sudo[44169]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:03.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:02 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:03.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 sudo[43977]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:00:03.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 sudo[43977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:00:03.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 sudo[43977]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:00:03.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 sudo[43977]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:00:04.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 ceph-mon[35365]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:04.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 ceph-mon[35365]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:04.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 ceph-mon[35365]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:04.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 ceph-mon[35365]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:04.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:04.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:04.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:04.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:03 smithi111 ceph-mon[35365]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:04.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:03 smithi019 ceph-mon[30938]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:04.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:03 smithi019 ceph-mon[30938]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:04.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:03 smithi019 ceph-mon[30938]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:04.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:03 smithi019 ceph-mon[30938]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:04.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:04.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:04.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:03 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:04.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:03 smithi019 ceph-mon[30938]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:04.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:03 smithi196 ceph-mon[35731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:04.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:03 smithi196 ceph-mon[35731]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:04.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:03 smithi196 ceph-mon[35731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:04.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:03 smithi196 ceph-mon[35731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:00:04.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:03 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:00:04.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:03 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:00:04.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:03 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:00:04.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:03 smithi196 ceph-mon[35731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:00:06.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:05 smithi111 ceph-mon[35365]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:06.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:05 smithi019 ceph-mon[30938]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:06.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:05 smithi196 ceph-mon[35731]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:08.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:07 smithi111 ceph-mon[35365]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:08.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:07 smithi019 ceph-mon[30938]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:08.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:07 smithi196 ceph-mon[35731]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:09.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:08 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:00:09.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:08 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:09.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:08 smithi111 ceph-mon[35365]: Deploying daemon osd.3 on smithi019 2024-09-19T04:00:09.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:08 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:00:09.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:08 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:09.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:08 smithi019 ceph-mon[30938]: Deploying daemon osd.3 on smithi019 2024-09-19T04:00:09.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:08 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:00:09.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:08 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:09.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:08 smithi196 ceph-mon[35731]: Deploying daemon osd.3 on smithi019 2024-09-19T04:00:10.039 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:09 smithi019 ceph-mon[30938]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:10.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:09 smithi111 ceph-mon[35365]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:10.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:09 smithi196 ceph-mon[35731]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:12.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:11 smithi111 ceph-mon[35365]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:12.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:11 smithi019 ceph-mon[30938]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:12.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:11 smithi196 ceph-mon[35731]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:14.131 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:13 smithi019 ceph-mon[30938]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:14.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:13 smithi111 ceph-mon[35365]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:14.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:13 smithi196 ceph-mon[35731]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:15.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:15.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:15.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:15.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:15.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:15.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:16.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:15 smithi019 ceph-mon[30938]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:16.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:15 smithi111 ceph-mon[35365]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:16.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:15 smithi196 ceph-mon[35731]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:17.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:17 smithi019 ceph-mon[30938]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:17.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:17.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:17 smithi196 ceph-mon[35731]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:17.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:17 smithi111 ceph-mon[35365]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.303+0000 7f8344ff9640 1 -- 172.21.15.19:0/177835959 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8348062630 con 0x7f83300a31b0 2024-09-19T04:00:18.305 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-09-19T04:00:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f83300a31b0 msgr2=0x7f83300a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f83300a31b0 0x7f83300a5670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f833c000c00 tx=0x7f833c0315d0 comp rx=0 tx=0).stop 2024-09-19T04:00:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348100b10 msgr2=0x7f834806b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348100b10 0x7f834806b330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f833800d770 tx=0x7f833800dc30 comp rx=0 tx=0).stop 2024-09-19T04:00:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 shutdown_connections 2024-09-19T04:00:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f83300a31b0 0x7f83300a5670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83481026d0 0x7f834819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8348101d10 0x7f834819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 --2- 172.21.15.19:0/177835959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8348100b10 0x7f834806b330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 >> 172.21.15.19:0/177835959 conn(0x7f83480fc2a0 msgr2=0x7f83481050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:00:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 shutdown_connections 2024-09-19T04:00:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:00:18.306+0000 7f834dbbb640 1 -- 172.21.15.19:0/177835959 wait complete. 2024-09-19T04:00:18.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:18 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:18.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:18 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:18 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.953 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:18 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:18.953 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:18 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.953 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:18 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:18 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:18.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:18 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:18 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:18.960 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T04:00:18.962 INFO:tasks.cephadm:Deploying osd.4 on smithi111 with /dev/vg_nvme/lv_3... 2024-09-19T04:00:18.962 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:00:19.217 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:00:20.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:20 smithi019 ceph-mon[30938]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:20.565 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:20 smithi111 ceph-mon[35365]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:20.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:20 smithi196 ceph-mon[35731]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:20.932 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:00:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3[51401]: 2024-09-19T04:00:20.441+0000 7f3be3a98640 -1 osd.3 0 log_to_monitors true 2024-09-19T04:00:21.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:21 smithi111 ceph-mon[35365]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:00:21.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:21 smithi019 ceph-mon[30938]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:00:21.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:21 smithi196 ceph-mon[35731]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:00:22.005 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T04:00:22.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2024-09-19T04:00:22.282 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:00:22.573 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:22.574 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:22.575 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.681 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3[51401]: 2024-09-19T04:00:22.322+0000 7f3bdfa1b640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T04:00:22.681 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3[51401]: 2024-09-19T04:00:22.328+0000 7f3bdb843640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:22.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:22.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:22.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:22.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:22.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:22.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 -- 172.21.15.111:0/2740091627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8074c50 msgr2=0x7f20a80750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:22.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 --2- 172.21.15.111:0/2740091627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8074c50 0x7f20a80750b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2090009900 tx=0x7f209002a410 comp rx=0 tx=0).stop 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 -- 172.21.15.111:0/2740091627 shutdown_connections 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 --2- 172.21.15.111:0/2740091627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a8109a80 0x7f20a810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 --2- 172.21.15.111:0/2740091627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8074c50 0x7f20a80750b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 --2- 172.21.15.111:0/2740091627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a8076220 0x7f20a8074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 -- 172.21.15.111:0/2740091627 >> 172.21.15.111:0/2740091627 conn(0x7f20a80fd390 msgr2=0x7f20a80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 -- 172.21.15.111:0/2740091627 shutdown_connections 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.770+0000 7f20acc18640 1 -- 172.21.15.111:0/2740091627 wait complete. 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20acc18640 1 Processor -- start 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20acc18640 1 -- start start 2024-09-19T04:00:22.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20acc18640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a8074c50 0x7f20a819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20acc18640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a8076220 0x7f20a819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20acc18640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8109a80 0x7f20a81a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20acc18640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810faa0 con 0x7f20a8074c50 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20acc18640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810f920 con 0x7f20a8109a80 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20acc18640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810fc20 con 0x7f20a8076220 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20a6d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8109a80 0x7f20a81a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20a6d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8109a80 0x7f20a81a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49444/0 (socket says 172.21.15.111:49444) 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20a6d9d640 1 -- 172.21.15.111:0/706882854 learned_addr learned my addr 172.21.15.111:0/706882854 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20a5d9b640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a8076220 0x7f20a819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.771+0000 7f20a659c640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a8074c50 0x7f20a819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f20a6d9d640 1 -- 172.21.15.111:0/706882854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a8076220 msgr2=0x7f20a819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:22.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f20a6d9d640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a8076220 0x7f20a819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f20a6d9d640 1 -- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a8074c50 msgr2=0x7f20a819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f20a6d9d640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a8074c50 0x7f20a819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f20a6d9d640 1 -- 172.21.15.111:0/706882854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a81a1d10 con 0x7f20a8109a80 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f20a6d9d640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8109a80 0x7f20a81a16a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f209800d750 tx=0x7f209800dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f209f7fe640 1 -- 172.21.15.111:0/706882854 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098017070 con 0x7f20a8109a80 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a81a1fa0 con 0x7f20a8109a80 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a81a2500 con 0x7f20a8109a80 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f209f7fe640 1 -- 172.21.15.111:0/706882854 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2098005500 con 0x7f20a8109a80 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.772+0000 7f209f7fe640 1 -- 172.21.15.111:0/706882854 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098020060 con 0x7f20a8109a80 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.773+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20880052c0 con 0x7f20a8109a80 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.774+0000 7f209f7fe640 1 -- 172.21.15.111:0/706882854 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2098009bd0 con 0x7f20a8109a80 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.775+0000 7f209f7fe640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f20940a3270 0x7f20940a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:00:22.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.775+0000 7f209f7fe640 1 -- 172.21.15.111:0/706882854 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f20980ce310 con 0x7f20a8109a80 2024-09-19T04:00:22.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.775+0000 7f20a659c640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f20940a3270 0x7f20940a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:00:22.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.776+0000 7f20a659c640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f20940a3270 0x7f20940a5730 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f20a8076600 tx=0x7f20a0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:00:22.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.778+0000 7f209f7fe640 1 -- 172.21.15.111:0/706882854 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f209809bbe0 con 0x7f20a8109a80 2024-09-19T04:00:22.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:22.950+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2088000bf0 con 0x7f20940a3270 2024-09-19T04:00:23.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi019 2024-09-19T04:00:23.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-19T04:00:23.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] boot 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:00:23.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:23 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:23.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi019 2024-09-19T04:00:23.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-19T04:00:23.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] boot 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:00:23.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:23 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:23.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi019 2024-09-19T04:00:23.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-19T04:00:23.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: from='osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: osd.3 [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] boot 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:00:23.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:23 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:24.606 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:24 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:00:24.607 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:24 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:00:24.607 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:24 smithi111 ceph-mon[35365]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:24.607 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:24 smithi111 ceph-mon[35365]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:00:24.607 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:24 smithi111 ceph-mon[35365]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:00:24.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:24 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:00:24.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:24 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:00:24.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:24 smithi019 ceph-mon[30938]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:24.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:24 smithi019 ceph-mon[30938]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:00:24.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:24 smithi019 ceph-mon[30938]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:00:24.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:24 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:00:24.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:24 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:00:24.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:24 smithi196 ceph-mon[35731]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:00:24.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:24 smithi196 ceph-mon[35731]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:00:24.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:24 smithi196 ceph-mon[35731]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:00:25.888 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:25 smithi111 ceph-mon[35365]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:25.888 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:25 smithi111 ceph-mon[35365]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:00:25.888 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:25 smithi111 ceph-mon[35365]: from='client.? 172.21.15.111:0/3155526286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]: dispatch 2024-09-19T04:00:25.889 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:25 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]: dispatch 2024-09-19T04:00:25.889 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:25 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]': finished 2024-09-19T04:00:25.889 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:25 smithi111 ceph-mon[35365]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:00:25.889 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:25 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:25.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:25 smithi019 ceph-mon[30938]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:25.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:25 smithi019 ceph-mon[30938]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:00:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:25 smithi019 ceph-mon[30938]: from='client.? 172.21.15.111:0/3155526286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]: dispatch 2024-09-19T04:00:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:25 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]: dispatch 2024-09-19T04:00:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:25 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]': finished 2024-09-19T04:00:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:25 smithi019 ceph-mon[30938]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:00:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:25 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:25 smithi196 ceph-mon[35731]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:25 smithi196 ceph-mon[35731]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:00:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:25 smithi196 ceph-mon[35731]: from='client.? 172.21.15.111:0/3155526286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]: dispatch 2024-09-19T04:00:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:25 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]: dispatch 2024-09-19T04:00:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:25 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa74ad7b-8602-4087-aee4-c6e9632864c5"}]': finished 2024-09-19T04:00:25.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:25 smithi196 ceph-mon[35731]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:00:25.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:25 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:26.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:26 smithi111 ceph-mon[35365]: from='client.? 172.21.15.111:0/680267469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:26.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:26 smithi111 ceph-mon[35365]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:00:26.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:26.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:26 smithi019 ceph-mon[30938]: from='client.? 172.21.15.111:0/680267469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:26.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:26 smithi019 ceph-mon[30938]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:00:26.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:26.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:26 smithi196 ceph-mon[35731]: from='client.? 172.21.15.111:0/680267469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:26.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:26 smithi196 ceph-mon[35731]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:00:26.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:28.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:28 smithi111 ceph-mon[35365]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:28.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:28 smithi019 ceph-mon[30938]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:28.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:28 smithi196 ceph-mon[35731]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:30 smithi111 ceph-mon[35365]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:30.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:30 smithi019 ceph-mon[30938]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:30.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:30 smithi196 ceph-mon[35731]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:32.634 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:32 smithi111 ceph-mon[35365]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:00:32.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:32 smithi019 ceph-mon[30938]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:00:32.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:32 smithi196 ceph-mon[35731]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:00:34.627 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:34 smithi111 ceph-mon[35365]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:00:34.627 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:00:34.627 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:34.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:34 smithi019 ceph-mon[30938]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:00:34.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:00:34.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:34.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:34 smithi196 ceph-mon[35731]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:00:34.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:00:34.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:35.389 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:35 smithi111 ceph-mon[35365]: Deploying daemon osd.4 on smithi111 2024-09-19T04:00:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:35 smithi019 ceph-mon[30938]: Deploying daemon osd.4 on smithi111 2024-09-19T04:00:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:35 smithi196 ceph-mon[35731]: Deploying daemon osd.4 on smithi111 2024-09-19T04:00:36.579 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:36 smithi111 ceph-mon[35365]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T04:00:36.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:36 smithi019 ceph-mon[30938]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T04:00:36.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:36 smithi196 ceph-mon[35731]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T04:00:37.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:37 smithi111 ceph-mon[35365]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T04:00:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:37 smithi019 ceph-mon[30938]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T04:00:37.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:37 smithi196 ceph-mon[35731]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T04:00:39.612 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:39 smithi111 ceph-mon[35365]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:00:39.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:39 smithi019 ceph-mon[30938]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:00:39.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:39 smithi196 ceph-mon[35731]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:00:40.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:40.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:40.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:40.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:40.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:40.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:42.532 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:42 smithi111 ceph-mon[35365]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:00:42.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:42 smithi019 ceph-mon[30938]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:00:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:42 smithi196 ceph-mon[35731]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:00:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:43 smithi111 ceph-mon[35365]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:43.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:43 smithi196 ceph-mon[35731]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:43.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:43.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:43.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:43.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:43.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:43 smithi019 ceph-mon[30938]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:44.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:44.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:44.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.297+0000 7f209f7fe640 1 -- 172.21.15.111:0/706882854 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2088000bf0 con 0x7f20940a3270 2024-09-19T04:00:44.299 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 4 on host 'smithi111' 2024-09-19T04:00:44.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f20940a3270 msgr2=0x7f20940a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:44.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f20940a3270 0x7f20940a5730 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f20a8076600 tx=0x7f20a0009680 comp rx=0 tx=0).stop 2024-09-19T04:00:44.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8109a80 msgr2=0x7f20a81a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:44.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8109a80 0x7f20a81a16a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f209800d750 tx=0x7f209800dc10 comp rx=0 tx=0).stop 2024-09-19T04:00:44.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 shutdown_connections 2024-09-19T04:00:44.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f20940a3270 0x7f20940a5730 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:44.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20a8109a80 0x7f20a81a16a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:44.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a8076220 0x7f20a819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:44.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 --2- 172.21.15.111:0/706882854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a8074c50 0x7f20a819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:44.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.299+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 >> 172.21.15.111:0/706882854 conn(0x7f20a80fd390 msgr2=0x7f20a81059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:00:44.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.300+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 shutdown_connections 2024-09-19T04:00:44.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:00:44.300+0000 7f20acc18640 1 -- 172.21.15.111:0/706882854 wait complete. 2024-09-19T04:00:44.928 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:44.928 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:44.929 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:44.956 DEBUG:teuthology.orchestra.run.smithi111:osd.4> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T04:00:44.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:44.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:44.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:44.959 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-09-19T04:00:44.959 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:00:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:00:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:45.209 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:00:46.338 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:46 smithi196 ceph-mon[35731]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:46.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:46 smithi111 ceph-mon[35365]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:46.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:46 smithi019 ceph-mon[30938]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:46.905 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:00:46 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:00:46.552+0000 7fa74db6e640 -1 osd.4 0 log_to_monitors true 2024-09-19T04:00:47.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:47 smithi111 ceph-mon[35365]: from='osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:00:47.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:47 smithi111 ceph-mon[35365]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:00:47.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:47 smithi196 ceph-mon[35731]: from='osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:00:47.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:47 smithi196 ceph-mon[35731]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:00:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:47 smithi019 ceph-mon[30938]: from='osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:00:47.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:47 smithi019 ceph-mon[30938]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:00:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T04:00:47.717 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-09-19T04:00:47.973 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:00:48.382 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:48.382 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:00:48.382 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:00:48.382 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:48.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:00:48.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:00:48.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:48.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:48.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:48.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:48.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/200030747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 msgr2=0x7f4b48104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:48.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/200030747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 0x7f4b48104a90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009900 tx=0x7f4b3002a410 comp rx=0 tx=0).stop 2024-09-19T04:00:48.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/200030747 shutdown_connections 2024-09-19T04:00:48.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/200030747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b48107900 0x7f4b48109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:48.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/200030747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b48104fd0 0x7f4b481073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:48.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/200030747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 0x7f4b48104a90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/200030747 >> 172.21.15.196:0/200030747 conn(0x7f4b480fc2a0 msgr2=0x7f4b480fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/200030747 shutdown_connections 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.476+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/200030747 wait complete. 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.477+0000 7f4b4d1bd640 1 Processor -- start 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.477+0000 7f4b4d1bd640 1 -- start start 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4d1bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 0x7f4b48197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4d1bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b48104fd0 0x7f4b48198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4d1bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b48107900 0x7f4b4819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4d1bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d8d0 con 0x7f4b48107900 2024-09-19T04:00:48.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4d1bd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d750 con 0x7f4b48104fd0 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4d1bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810da50 con 0x7f4b481026a0 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 0x7f4b48197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 0x7f4b48197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43718/0 (socket says 172.21.15.196:43718) 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 -- 172.21.15.196:0/601816952 learned_addr learned my addr 172.21.15.196:0/601816952 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4659c640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b48104fd0 0x7f4b48198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4759e640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b48107900 0x7f4b4819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 -- 172.21.15.196:0/601816952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b48104fd0 msgr2=0x7f4b48198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b48104fd0 0x7f4b48198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 -- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b48107900 msgr2=0x7f4b4819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:00:48.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b48107900 0x7f4b4819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 -- 172.21.15.196:0/601816952 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4819fb60 con 0x7f4b481026a0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b46d9d640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 0x7f4b48197f30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009230 tx=0x7f4b30004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b3ffff640 1 -- 172.21.15.196:0/601816952 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3003f030 con 0x7f4b481026a0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4819fdf0 con 0x7f4b481026a0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b3ffff640 1 -- 172.21.15.196:0/601816952 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b30003ce0 con 0x7f4b481026a0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.478+0000 7f4b3ffff640 1 -- 172.21.15.196:0/601816952 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30035070 con 0x7f4b481026a0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.479+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b481a0240 con 0x7f4b481026a0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.480+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b480fdd90 con 0x7f4b481026a0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.481+0000 7f4b3ffff640 1 -- 172.21.15.196:0/601816952 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4b30042030 con 0x7f4b481026a0 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.481+0000 7f4b3ffff640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4b340a3270 0x7f4b340a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:00:48.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.482+0000 7f4b3ffff640 1 -- 172.21.15.196:0/601816952 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f4b300eece0 con 0x7f4b481026a0 2024-09-19T04:00:48.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.482+0000 7f4b4659c640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4b340a3270 0x7f4b340a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:00:48.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.482+0000 7f4b4659c640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4b340a3270 0x7f4b340a5730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b48199450 tx=0x7f4b3800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:00:48.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.486+0000 7f4b3ffff640 1 -- 172.21.15.196:0/601816952 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4b300bc540 con 0x7f4b481026a0 2024-09-19T04:00:48.655 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:00:48.316+0000 7fa749af1640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T04:00:48.655 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:00:48.322+0000 7fa745919640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:00:48.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:48.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:48.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:00:48.660+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4b48062630 con 0x7f4b340a3270 2024-09-19T04:00:48.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:48.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:00:48.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:00:48.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:00:49.577 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi111 2024-09-19T04:00:49.577 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-19T04:00:49.577 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:00:49.577 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T04:00:49.577 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:49.578 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:49.578 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:00:49.578 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:00:49.578 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:49.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi111 2024-09-19T04:00:49.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-19T04:00:49.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:00:49.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T04:00:49.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:49.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:49.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:00:49.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:00:49.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:49.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi111 2024-09-19T04:00:49.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-19T04:00:49.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:00:49.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T04:00:49.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:49.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:49.688 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:00:49.688 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:00:49.688 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:50.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:50 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:00:50.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:50 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:00:50.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:50 smithi196 ceph-mon[35731]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:50.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:50 smithi196 ceph-mon[35731]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:00:50.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:50 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:50.509 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:50 smithi196 ceph-mon[35731]: osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] boot 2024-09-19T04:00:50.509 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:50 smithi196 ceph-mon[35731]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:00:50.509 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:50 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:50 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:00:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:50 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:00:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:50 smithi111 ceph-mon[35365]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:50 smithi111 ceph-mon[35365]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:00:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:50 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:50 smithi111 ceph-mon[35365]: osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] boot 2024-09-19T04:00:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:50 smithi111 ceph-mon[35365]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:00:50.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:50 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:50.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:50 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:00:50.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:50 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:00:50.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:50 smithi019 ceph-mon[30938]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T04:00:50.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:50 smithi019 ceph-mon[30938]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:00:50.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:50 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:50.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:50 smithi019 ceph-mon[30938]: osd.4 [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] boot 2024-09-19T04:00:50.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:50 smithi019 ceph-mon[30938]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:00:50.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:50 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:00:51.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:51 smithi196 ceph-mon[35731]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T04:00:51.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:51 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/2712300235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]: dispatch 2024-09-19T04:00:51.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:51 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]: dispatch 2024-09-19T04:00:51.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:51 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]': finished 2024-09-19T04:00:51.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:51 smithi196 ceph-mon[35731]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:00:51.550 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:51 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:00:51.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:51 smithi111 ceph-mon[35365]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T04:00:51.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:51 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/2712300235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]: dispatch 2024-09-19T04:00:51.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:51 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]: dispatch 2024-09-19T04:00:51.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:51 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]': finished 2024-09-19T04:00:51.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:51 smithi111 ceph-mon[35365]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:00:51.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:51 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:00:51.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:51 smithi019 ceph-mon[30938]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T04:00:51.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:51 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/2712300235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]: dispatch 2024-09-19T04:00:51.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:51 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]: dispatch 2024-09-19T04:00:51.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:51 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14c144ac-b587-4412-8aae-b0aef6058124"}]': finished 2024-09-19T04:00:51.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:51 smithi019 ceph-mon[30938]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:00:51.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:51 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:00:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:52 smithi111 ceph-mon[35365]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:52 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/89466411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:52 smithi019 ceph-mon[30938]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:52 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/89466411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:52.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:52 smithi196 ceph-mon[35731]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:52.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:52 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/89466411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:00:53.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:53 smithi111 ceph-mon[35365]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:53.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:53 smithi019 ceph-mon[30938]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:53.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:53 smithi196 ceph-mon[35731]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:55.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:55 smithi019 ceph-mon[30938]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:55.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:55 smithi196 ceph-mon[35731]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:55.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:55 smithi111 ceph-mon[35365]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:57.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:57 smithi196 ceph-mon[35731]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:57.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:57 smithi019 ceph-mon[30938]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:57.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:57 smithi111 ceph-mon[35365]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:59 smithi019 ceph-mon[30938]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:00:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:00:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:59 smithi196 ceph-mon[35731]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:00:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:00:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:00:59.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:59 smithi111 ceph-mon[35365]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:00:59.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:00:59.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:00:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:00.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:00 smithi111 ceph-mon[35365]: Deploying daemon osd.5 on smithi196 2024-09-19T04:01:00.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:00 smithi019 ceph-mon[30938]: Deploying daemon osd.5 on smithi196 2024-09-19T04:01:00.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:00 smithi196 ceph-mon[35731]: Deploying daemon osd.5 on smithi196 2024-09-19T04:01:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:01 smithi196 ceph-mon[35731]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:01 smithi111 ceph-mon[35365]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:01.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:01 smithi019 ceph-mon[30938]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:03.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:03 smithi196 ceph-mon[35731]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:03.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:03 smithi111 ceph-mon[35365]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:03.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:03 smithi019 ceph-mon[30938]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:05 smithi196 ceph-mon[35731]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:05.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:05 smithi019 ceph-mon[30938]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:05 smithi111 ceph-mon[35365]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:06 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:06 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:06 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:06 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:06 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:06 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:06 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:06 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:06 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:07.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:07 smithi019 ceph-mon[30938]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:07.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:07 smithi196 ceph-mon[35731]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:08.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:07 smithi111 ceph-mon[35365]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:09.633 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:09 smithi196 ceph-mon[35731]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:09.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:09.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:09.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:09.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:09.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:09 smithi111 ceph-mon[35365]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:09.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.673+0000 7f4b3ffff640 1 -- 172.21.15.196:0/601816952 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4b48062630 con 0x7f4b340a3270 2024-09-19T04:01:09.674 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-09-19T04:01:09.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4b340a3270 msgr2=0x7f4b340a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4b340a3270 0x7f4b340a5730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b48199450 tx=0x7f4b3800b390 comp rx=0 tx=0).stop 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 msgr2=0x7f4b48197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 0x7f4b48197f30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009230 tx=0x7f4b30004360 comp rx=0 tx=0).stop 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 shutdown_connections 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4b340a3270 0x7f4b340a5730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b48107900 0x7f4b4819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b48104fd0 0x7f4b48198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 --2- 172.21.15.196:0/601816952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b481026a0 0x7f4b48197f30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 >> 172.21.15.196:0/601816952 conn(0x7f4b480fc2a0 msgr2=0x7f4b48103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 shutdown_connections 2024-09-19T04:01:09.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:01:09.675+0000 7f4b4d1bd640 1 -- 172.21.15.196:0/601816952 wait complete. 2024-09-19T04:01:09.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:09 smithi019 ceph-mon[30938]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:09.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:09.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:09.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:09.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:09.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:10.262 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T04:01:10.264 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-09-19T04:01:10.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:01:10.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:10 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:10.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:10 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:10.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:10 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:01:10.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:10 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:10.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:10 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:10.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:10 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:10.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:10 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:10.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:10 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:10.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:10 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:11.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:11 smithi019 ceph-mon[30938]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:11.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:11 smithi196 ceph-mon[35731]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:12.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:11 smithi111 ceph-mon[35365]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:12.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:01:12 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:01:12.033+0000 7fb23396c640 -1 osd.5 0 log_to_monitors true 2024-09-19T04:01:12.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:12 smithi019 ceph-mon[30938]: from='osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:01:12.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:12 smithi019 ceph-mon[30938]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:01:12.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:12 smithi196 ceph-mon[35731]: from='osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:01:12.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:12 smithi196 ceph-mon[35731]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:01:13.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:12 smithi111 ceph-mon[35365]: from='osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:01:13.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:12 smithi111 ceph-mon[35365]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:01:13.319 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:01:13.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-09-19T04:01:13.601 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:01:13.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:13.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:01:13.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi196 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:13.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:13.960 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi196 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:14.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:14.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-19T04:01:14.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:14.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:14.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 -- 172.21.15.19:0/528493185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 msgr2=0x7f593c10a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 --2- 172.21.15.19:0/528493185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 0x7f593c10a060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5934009ca0 tx=0x7f593402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 -- 172.21.15.19:0/528493185 shutdown_connections 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 --2- 172.21.15.19:0/528493185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 0x7f593c10a060 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 --2- 172.21.15.19:0/528493185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f593c102e10 0x7f593c103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 --2- 172.21.15.19:0/528493185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f593c101c10 0x7f593c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 -- 172.21.15.19:0/528493185 >> 172.21.15.19:0/528493185 conn(0x7f593c0fd3c0 msgr2=0x7f593c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 -- 172.21.15.19:0/528493185 shutdown_connections 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.088+0000 7f5940fb8640 1 -- 172.21.15.19:0/528493185 wait complete. 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5940fb8640 1 Processor -- start 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5940fb8640 1 -- start start 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5940fb8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f593c101c10 0x7f593c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:01:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5940fb8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f593c102e10 0x7f593c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5940fb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 0x7f593c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5940fb8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10d910 con 0x7f593c1037d0 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5940fb8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10d790 con 0x7f593c101c10 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5940fb8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10da90 con 0x7f593c102e10 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f593ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 0x7f593c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f593ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 0x7f593c19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56580/0 (socket says 172.21.15.19:56580) 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f593ad9d640 1 -- 172.21.15.19:0/2138302028 learned_addr learned my addr 172.21.15.19:0/2138302028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.089+0000 7f5939d9b640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f593c102e10 0x7f593c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f593a59c640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f593c101c10 0x7f593c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f593ad9d640 1 -- 172.21.15.19:0/2138302028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f593c102e10 msgr2=0x7f593c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f593ad9d640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f593c102e10 0x7f593c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f593ad9d640 1 -- 172.21.15.19:0/2138302028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f593c101c10 msgr2=0x7f593c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f593ad9d640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f593c101c10 0x7f593c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f593ad9d640 1 -- 172.21.15.19:0/2138302028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c19fb70 con 0x7f593c1037d0 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f593a59c640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f593c101c10 0x7f593c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f593ad9d640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 0x7f593c19f500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5934005e30 tx=0x7f5934031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f592b7fe640 1 -- 172.21.15.19:0/2138302028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934003c30 con 0x7f593c1037d0 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f592b7fe640 1 -- 172.21.15.19:0/2138302028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5934003dd0 con 0x7f593c1037d0 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c19fe00 con 0x7f593c1037d0 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f592b7fe640 1 -- 172.21.15.19:0/2138302028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593402fde0 con 0x7f593c1037d0 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.090+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1a0330 con 0x7f593c1037d0 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.091+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593c109010 con 0x7f593c1037d0 2024-09-19T04:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.092+0000 7f592b7fe640 1 -- 172.21.15.19:0/2138302028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5934004050 con 0x7f593c1037d0 2024-09-19T04:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.093+0000 7f592b7fe640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f59240a3270 0x7f59240a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.093+0000 7f593a59c640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f59240a3270 0x7f59240a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.093+0000 7f592b7fe640 1 -- 172.21.15.19:0/2138302028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7f59340efdd0 con 0x7f593c1037d0 2024-09-19T04:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.094+0000 7f593a59c640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f59240a3270 0x7f59240a5730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f592c006fd0 tx=0x7f592c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.096+0000 7f592b7fe640 1 -- 172.21.15.19:0/2138302028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59340bdad0 con 0x7f593c1037d0 2024-09-19T04:01:14.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:14.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:01:14.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:01:14.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi196 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:14.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:14.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:14.282+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f593c062630 con 0x7f59240a3270 2024-09-19T04:01:14.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:01:14.066+0000 7fb22f8ef640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T04:01:14.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:01:14.073+0000 7fb22770f640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:01:14.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:01:14.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:01:14.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:14.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:14.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='osd.5 ' entity='osd.5' 2024-09-19T04:01:14.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:01:14.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:01:14.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:01:14.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:14.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] boot 2024-09-19T04:01:14.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:01:14.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:14.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:01:14.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:01:14.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:14.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:14.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='osd.5 ' entity='osd.5' 2024-09-19T04:01:14.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:01:14.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:01:14.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:01:14.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:14.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] boot 2024-09-19T04:01:14.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:01:14.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:01:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:01:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='osd.5 ' entity='osd.5' 2024-09-19T04:01:15.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:01:15.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:01:15.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:01:15.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:15.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: osd.5 [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] boot 2024-09-19T04:01:15.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:01:15.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:01:15.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:15 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:01:15.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:15 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:01:15.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:15 smithi196 ceph-mon[35731]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:15.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:15 smithi196 ceph-mon[35731]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:01:16.052 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:15 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:01:16.053 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:15 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:01:16.053 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:15 smithi019 ceph-mon[30938]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:16.053 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:15 smithi019 ceph-mon[30938]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:01:16.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:15 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:01:16.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:15 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:01:16.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:15 smithi111 ceph-mon[35365]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:01:16.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:15 smithi111 ceph-mon[35365]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:01:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:01:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3680199984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "531057c9-5241-4e2e-9f6a-5ce00fb4cd26"}]: dispatch 2024-09-19T04:01:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3680199984' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "531057c9-5241-4e2e-9f6a-5ce00fb4cd26"}]': finished 2024-09-19T04:01:17.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:01:17.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:17.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1991914558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:01:17.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:01:17.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:01:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3680199984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "531057c9-5241-4e2e-9f6a-5ce00fb4cd26"}]: dispatch 2024-09-19T04:01:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3680199984' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "531057c9-5241-4e2e-9f6a-5ce00fb4cd26"}]': finished 2024-09-19T04:01:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:01:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1991914558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:01:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:01:18.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:18.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:18.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:01:18.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3680199984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "531057c9-5241-4e2e-9f6a-5ce00fb4cd26"}]: dispatch 2024-09-19T04:01:18.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3680199984' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "531057c9-5241-4e2e-9f6a-5ce00fb4cd26"}]': finished 2024-09-19T04:01:18.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T04:01:18.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:18.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1991914558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:01:18.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:01:18.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:19.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:19 smithi111 ceph-mon[35365]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:19.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:19 smithi111 ceph-mon[35365]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:01:19.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:19 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:19.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:19 smithi019 ceph-mon[30938]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:19.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:19 smithi019 ceph-mon[30938]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:01:19.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:19.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:19 smithi196 ceph-mon[35731]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:19.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:19 smithi196 ceph-mon[35731]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:01:19.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:21 smithi111 ceph-mon[35365]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:01:21.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:21 smithi019 ceph-mon[30938]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:01:21.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:21 smithi196 ceph-mon[35731]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:01:23.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:23 smithi019 ceph-mon[30938]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T04:01:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:23 smithi111 ceph-mon[35365]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T04:01:23.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:23 smithi196 ceph-mon[35731]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T04:01:25.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:25 smithi111 ceph-mon[35365]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-19T04:01:25.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:25 smithi019 ceph-mon[30938]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-19T04:01:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:25 smithi196 ceph-mon[35731]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-19T04:01:26.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:01:26.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:26.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:26 smithi019 ceph-mon[30938]: Deploying daemon osd.6 on smithi019 2024-09-19T04:01:26.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:01:26.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:26.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:26 smithi111 ceph-mon[35365]: Deploying daemon osd.6 on smithi019 2024-09-19T04:01:26.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:01:26.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:26.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:26 smithi196 ceph-mon[35731]: Deploying daemon osd.6 on smithi019 2024-09-19T04:01:27.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:27 smithi019 ceph-mon[30938]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:01:27.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:27 smithi111 ceph-mon[35365]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:01:27.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:27 smithi196 ceph-mon[35731]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T04:01:29.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:29 smithi019 ceph-mon[30938]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:01:29.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:29 smithi111 ceph-mon[35365]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:01:29.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:29 smithi196 ceph-mon[35731]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:01:31.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:31 smithi111 ceph-mon[35365]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:31.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:31 smithi019 ceph-mon[30938]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:31.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:31 smithi196 ceph-mon[35731]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:32.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:32.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:32.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:32.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:32.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:32.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:33.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:33 smithi019 ceph-mon[30938]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:34.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:33 smithi111 ceph-mon[35365]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:34.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:33 smithi196 ceph-mon[35731]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:35 smithi196 ceph-mon[35731]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:35.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:35.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:35 smithi019 ceph-mon[30938]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:35.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:35.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:35.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:35.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:35.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:35.878 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-09-19T04:01:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.875+0000 7f592b7fe640 1 -- 172.21.15.19:0/2138302028 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f593c062630 con 0x7f59240a3270 2024-09-19T04:01:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f59240a3270 msgr2=0x7f59240a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f59240a3270 0x7f59240a5730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f592c006fd0 tx=0x7f592c008040 comp rx=0 tx=0).stop 2024-09-19T04:01:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 msgr2=0x7f593c19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 0x7f593c19f500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5934005e30 tx=0x7f5934031ee0 comp rx=0 tx=0).stop 2024-09-19T04:01:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 shutdown_connections 2024-09-19T04:01:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f59240a3270 0x7f59240a5730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f593c1037d0 0x7f593c19f500 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f593c102e10 0x7f593c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 --2- 172.21.15.19:0/2138302028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f593c101c10 0x7f593c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 >> 172.21.15.19:0/2138302028 conn(0x7f593c0fd3c0 msgr2=0x7f593c1061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:01:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 shutdown_connections 2024-09-19T04:01:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:01:35.878+0000 7f5940fb8640 1 -- 172.21.15.19:0/2138302028 wait complete. 2024-09-19T04:01:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:35 smithi111 ceph-mon[35365]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:36.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:36 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:36.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:36 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:36.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:36 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:36.681 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T04:01:36.684 INFO:tasks.cephadm:Deploying osd.7 on smithi111 with /dev/vg_nvme/lv_2... 2024-09-19T04:01:36.684 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:01:36.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:36 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:36.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:36 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:36.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:36 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:36.711 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:36 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:36.711 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:36 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:36.712 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:36 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:36.940 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:01:37.682 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:01:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:01:37.608+0000 7f6881454640 -1 osd.6 0 log_to_monitors true 2024-09-19T04:01:38.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:37 smithi019 ceph-mon[30938]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:38.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:37 smithi019 ceph-mon[30938]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:01:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:37 smithi111 ceph-mon[35365]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:38.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:37 smithi111 ceph-mon[35365]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:01:38.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:37 smithi196 ceph-mon[35731]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:38.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:37 smithi196 ceph-mon[35731]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:01:39.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:38 smithi111 ceph-mon[35365]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:01:39.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:38 smithi111 ceph-mon[35365]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:01:39.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:38 smithi111 ceph-mon[35365]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:01:39.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:38 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:39.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:38 smithi019 ceph-mon[30938]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:01:39.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:38 smithi019 ceph-mon[30938]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:01:39.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:38 smithi019 ceph-mon[30938]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:01:39.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:38 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:39.182 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:01:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:01:38.888+0000 7f687d3d7640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T04:01:39.182 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:01:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:01:38.902+0000 7f68791ff640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:01:39.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:38 smithi196 ceph-mon[35731]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:01:39.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:38 smithi196 ceph-mon[35731]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:01:39.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:38 smithi196 ceph-mon[35731]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:01:39.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:38 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:39.632 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T04:01:39.659 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2024-09-19T04:01:39.961 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:01:40.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:40.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:01:40.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T04:01:40.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:40.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:40.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.157 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.158 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:40.158 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:40.158 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:40.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:40.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:40.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:01:40.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T04:01:40.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:01:40.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:40.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 -- 172.21.15.111:0/1529895080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 msgr2=0x7f4274103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:40.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 --2- 172.21.15.111:0/1529895080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 0x7f4274103280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f426802a410 comp rx=0 tx=0).stop 2024-09-19T04:01:40.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 -- 172.21.15.111:0/1529895080 shutdown_connections 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 --2- 172.21.15.111:0/1529895080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42741037c0 0x7f427410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 --2- 172.21.15.111:0/1529895080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 0x7f4274103280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 --2- 172.21.15.111:0/1529895080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4274101c00 0x7f4274102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 -- 172.21.15.111:0/1529895080 >> 172.21.15.111:0/1529895080 conn(0x7f42740fd390 msgr2=0x7f42740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 -- 172.21.15.111:0/1529895080 shutdown_connections 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.441+0000 7f427a40b640 1 -- 172.21.15.111:0/1529895080 wait complete. 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f427a40b640 1 Processor -- start 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f427a40b640 1 -- start start 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f427a40b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274101c00 0x7f4274197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f427a40b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 0x7f42741984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:01:40.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f427a40b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42741037c0 0x7f427419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f427a40b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f427410d930 con 0x7f42741037c0 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f427a40b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f427410d7b0 con 0x7f4274102e00 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f427a40b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f427410dab0 con 0x7f4274101c00 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f42737fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 0x7f42741984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f42737fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 0x7f42741984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52756/0 (socket says 172.21.15.111:52756) 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.442+0000 7f42737fe640 1 -- 172.21.15.111:0/1298718349 learned_addr learned my addr 172.21.15.111:0/1298718349 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f4273fff640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274101c00 0x7f4274197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42789a8640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42741037c0 0x7f427419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42737fe640 1 -- 172.21.15.111:0/1298718349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274101c00 msgr2=0x7f4274197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42737fe640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274101c00 0x7f4274197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42737fe640 1 -- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42741037c0 msgr2=0x7f427419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:01:40.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42737fe640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42741037c0 0x7f427419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42737fe640 1 -- 172.21.15.111:0/1298718349 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427419fbe0 con 0x7f4274102e00 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42789a8640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42741037c0 0x7f427419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42737fe640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 0x7f42741984f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4268009230 tx=0x7f4268004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42717fa640 1 -- 172.21.15.111:0/1298718349 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426803f030 con 0x7f4274102e00 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42717fa640 1 -- 172.21.15.111:0/1298718349 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4268003ce0 con 0x7f4274102e00 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427419fe70 con 0x7f4274102e00 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f42717fa640 1 -- 172.21.15.111:0/1298718349 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268035070 con 0x7f4274102e00 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.443+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42741a0340 con 0x7f4274102e00 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.444+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4274108fa0 con 0x7f4274102e00 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.445+0000 7f42717fa640 1 -- 172.21.15.111:0/1298718349 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4268042030 con 0x7f4274102e00 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.446+0000 7f42717fa640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f425c0a32f0 0x7f425c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:01:40.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.446+0000 7f42717fa640 1 -- 172.21.15.111:0/1298718349 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f42680ef710 con 0x7f4274102e00 2024-09-19T04:01:40.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.446+0000 7f4273fff640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f425c0a32f0 0x7f425c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:01:40.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.447+0000 7f4273fff640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f425c0a32f0 0x7f425c0a57b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4274102c60 tx=0x7f4264009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:01:40.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.449+0000 7f42717fa640 1 -- 172.21.15.111:0/1298718349 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f42680bd0a0 con 0x7f4274102e00 2024-09-19T04:01:40.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:01:40.623+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4274062630 con 0x7f425c0a32f0 2024-09-19T04:01:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:01:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:01:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi019 2024-09-19T04:01:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-19T04:01:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] boot 2024-09-19T04:01:41.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:01:41.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:41.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:01:41.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:01:41.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:01:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:01:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi019 2024-09-19T04:01:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-19T04:01:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:41.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] boot 2024-09-19T04:01:41.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:01:41.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:41.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:01:41.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:01:41.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:01:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:01:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi019 2024-09-19T04:01:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-19T04:01:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: osd.6 [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] boot 2024-09-19T04:01:41.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:01:41.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:01:41.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:01:41.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:01:41.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:42.106 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:41 smithi111 ceph-mon[35365]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:42.107 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:41 smithi111 ceph-mon[35365]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:01:42.107 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:41 smithi111 ceph-mon[35365]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T04:01:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:41 smithi019 ceph-mon[30938]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:41 smithi019 ceph-mon[30938]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:01:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:41 smithi019 ceph-mon[30938]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T04:01:42.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:41 smithi196 ceph-mon[35731]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:01:42.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:41 smithi196 ceph-mon[35731]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:01:42.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:41 smithi196 ceph-mon[35731]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T04:01:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:43 smithi111 ceph-mon[35365]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:43 smithi111 ceph-mon[35365]: from='client.? 172.21.15.111:0/909755526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]: dispatch 2024-09-19T04:01:44.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:43 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]: dispatch 2024-09-19T04:01:44.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:43 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]': finished 2024-09-19T04:01:44.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:43 smithi111 ceph-mon[35365]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:01:44.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:01:44.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:43 smithi111 ceph-mon[35365]: from='client.? 172.21.15.111:0/1863249966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:01:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:43 smithi019 ceph-mon[30938]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:43 smithi019 ceph-mon[30938]: from='client.? 172.21.15.111:0/909755526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]: dispatch 2024-09-19T04:01:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:43 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]: dispatch 2024-09-19T04:01:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:43 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]': finished 2024-09-19T04:01:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:43 smithi019 ceph-mon[30938]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:01:44.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:01:44.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:43 smithi019 ceph-mon[30938]: from='client.? 172.21.15.111:0/1863249966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:01:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:43 smithi196 ceph-mon[35731]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:43 smithi196 ceph-mon[35731]: from='client.? 172.21.15.111:0/909755526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]: dispatch 2024-09-19T04:01:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:43 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]: dispatch 2024-09-19T04:01:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:43 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44f8f0de-ffc4-4b29-b40a-73b485a3c478"}]': finished 2024-09-19T04:01:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:43 smithi196 ceph-mon[35731]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:01:44.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:01:44.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:43 smithi196 ceph-mon[35731]: from='client.? 172.21.15.111:0/1863249966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:01:46.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:45 smithi019 ceph-mon[30938]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:45 smithi196 ceph-mon[35731]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:46.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:45 smithi111 ceph-mon[35365]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:48.063 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:47 smithi111 ceph-mon[35365]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:48.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:47 smithi019 ceph-mon[30938]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:48.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:47 smithi196 ceph-mon[35731]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:50.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:49 smithi111 ceph-mon[35365]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:50.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:49 smithi019 ceph-mon[30938]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:50.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:49 smithi196 ceph-mon[35731]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:52.174 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:51 smithi111 ceph-mon[35365]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:52.175 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:51 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:01:52.175 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:51 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:51 smithi019 ceph-mon[30938]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:51 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:01:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:51 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:52.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:51 smithi196 ceph-mon[35731]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:52.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:51 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:01:52.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:51 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:01:53.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:52 smithi019 ceph-mon[30938]: Deploying daemon osd.7 on smithi111 2024-09-19T04:01:53.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:52 smithi196 ceph-mon[35731]: Deploying daemon osd.7 on smithi111 2024-09-19T04:01:53.213 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:52 smithi111 ceph-mon[35365]: Deploying daemon osd.7 on smithi111 2024-09-19T04:01:54.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:53 smithi019 ceph-mon[30938]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:54.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:53 smithi196 ceph-mon[35731]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:54.245 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:53 smithi111 ceph-mon[35365]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:56.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:55 smithi019 ceph-mon[30938]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:56.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:55 smithi196 ceph-mon[35731]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:56.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:55 smithi111 ceph-mon[35365]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:58.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:57 smithi019 ceph-mon[30938]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:58.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:58.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:58.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:58.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:57 smithi196 ceph-mon[35731]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:58.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:58.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:58.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:58.260 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:57 smithi111 ceph-mon[35365]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:01:58.260 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:01:58.260 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:01:58.260 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:00.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:01:59 smithi111 ceph-mon[35365]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:00.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:01:59 smithi196 ceph-mon[35731]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:01:59 smithi019 ceph-mon[30938]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:01.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.292+0000 7f42717fa640 1 -- 172.21.15.111:0/1298718349 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4274062630 con 0x7f425c0a32f0 2024-09-19T04:02:01.294 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 7 on host 'smithi111' 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f425c0a32f0 msgr2=0x7f425c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f425c0a32f0 0x7f425c0a57b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4274102c60 tx=0x7f4264009110 comp rx=0 tx=0).stop 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 msgr2=0x7f42741984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 0x7f42741984f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4268009230 tx=0x7f4268004430 comp rx=0 tx=0).stop 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 shutdown_connections 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f425c0a32f0 0x7f425c0a57b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42741037c0 0x7f427419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4274102e00 0x7f42741984f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 --2- 172.21.15.111:0/1298718349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274101c00 0x7f4274197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 >> 172.21.15.111:0/1298718349 conn(0x7f42740fd390 msgr2=0x7f4274106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 shutdown_connections 2024-09-19T04:02:01.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:01.294+0000 7f427a40b640 1 -- 172.21.15.111:0/1298718349 wait complete. 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:01.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:01.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:01.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:01.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:01.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:01.976 DEBUG:teuthology.orchestra.run.smithi111:osd.7> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T04:02:01.979 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-09-19T04:02:01.979 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:02:02.232 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:02:03.537 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:02:03 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:02:03.512+0000 7f69a2e6e640 -1 osd.7 0 log_to_monitors true 2024-09-19T04:02:03.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:03 smithi196 ceph-mon[35731]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:03.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:03 smithi196 ceph-mon[35731]: from='osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:02:03.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:03 smithi196 ceph-mon[35731]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:02:03.791 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:03 smithi111 ceph-mon[35365]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:03.791 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:03 smithi111 ceph-mon[35365]: from='osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:02:03.791 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:03 smithi111 ceph-mon[35365]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:02:03.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:03 smithi019 ceph-mon[30938]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:03.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:03 smithi019 ceph-mon[30938]: from='osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:02:03.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:03 smithi019 ceph-mon[30938]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:02:04.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:04 smithi111 ceph-mon[35365]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:02:04.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:04 smithi111 ceph-mon[35365]: from='osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:02:04.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:04 smithi111 ceph-mon[35365]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:02:04.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:04 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:04.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:04 smithi111 ceph-mon[35365]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:02:04.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:04 smithi111 ceph-mon[35365]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:02:04.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:04 smithi196 ceph-mon[35731]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:02:04.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:04 smithi196 ceph-mon[35731]: from='osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:02:04.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:04 smithi196 ceph-mon[35731]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:02:04.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:04 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:04.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:04 smithi196 ceph-mon[35731]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:02:04.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:04 smithi196 ceph-mon[35731]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:02:04.855 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T04:02:04.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-09-19T04:02:04.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:04 smithi019 ceph-mon[30938]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:02:04.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:04 smithi019 ceph-mon[30938]: from='osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:02:04.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:04 smithi019 ceph-mon[30938]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:02:04.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:04 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:04.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:04 smithi019 ceph-mon[30938]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:02:04.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:04 smithi019 ceph-mon[30938]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:02:05.062 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:02:04 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:02:04.928+0000 7f699edf1640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T04:02:05.062 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:02:04 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:02:04.938+0000 7f6996c11640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:02:05.138 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:02:05.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 -- 172.21.15.196:0/572284290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1013f0 msgr2=0x7f8a7c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:05.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 --2- 172.21.15.196:0/572284290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1013f0 0x7f8a7c1037e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009900 tx=0x7f8a7802a410 comp rx=0 tx=0).stop 2024-09-19T04:02:05.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 -- 172.21.15.196:0/572284290 shutdown_connections 2024-09-19T04:02:05.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 --2- 172.21.15.196:0/572284290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a7c1067d0 0x7f8a7c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:05.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 --2- 172.21.15.196:0/572284290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a7c103d20 0x7f8a7c106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:05.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 --2- 172.21.15.196:0/572284290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1013f0 0x7f8a7c1037e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 -- 172.21.15.196:0/572284290 >> 172.21.15.196:0/572284290 conn(0x7f8a7c0fb610 msgr2=0x7f8a7c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 -- 172.21.15.196:0/572284290 shutdown_connections 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.608+0000 7f8a84019640 1 -- 172.21.15.196:0/572284290 wait complete. 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a84019640 1 Processor -- start 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a84019640 1 -- start start 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a84019640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a7c1013f0 0x7f8a7c100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a84019640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a7c103d20 0x7f8a7c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a84019640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1067d0 0x7f8a7c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a84019640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c10c780 con 0x7f8a7c103d20 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a84019640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c10c600 con 0x7f8a7c1013f0 2024-09-19T04:02:05.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a84019640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c10c900 con 0x7f8a7c1067d0 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a825b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1067d0 0x7f8a7c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a825b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1067d0 0x7f8a7c19f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:60598/0 (socket says 172.21.15.196:60598) 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a825b6640 1 -- 172.21.15.196:0/1715783383 learned_addr learned my addr 172.21.15.196:0/1715783383 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a81db5640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a7c1013f0 0x7f8a7c100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a825b6640 1 -- 172.21.15.196:0/1715783383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a7c1013f0 msgr2=0x7f8a7c100c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a825b6640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a7c1013f0 0x7f8a7c100c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a825b6640 1 -- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a7c103d20 msgr2=0x7f8a7c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a815b4640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a7c103d20 0x7f8a7c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a825b6640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a7c103d20 0x7f8a7c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a825b6640 1 -- 172.21.15.196:0/1715783383 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7c19fbd0 con 0x7f8a7c1067d0 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.609+0000 7f8a81db5640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a7c1013f0 0x7f8a7c100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:02:05.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.610+0000 7f8a825b6640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1067d0 0x7f8a7c19f2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400cad0 tx=0x7f8a7400cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.610+0000 7f8a72ffd640 1 -- 172.21.15.196:0/1715783383 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400ae20 con 0x7f8a7c1067d0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.610+0000 7f8a72ffd640 1 -- 172.21.15.196:0/1715783383 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a74005020 con 0x7f8a7c1067d0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.610+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7c19fe60 con 0x7f8a7c1067d0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.610+0000 7f8a72ffd640 1 -- 172.21.15.196:0/1715783383 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400f6f0 con 0x7f8a7c1067d0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.610+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c1a0320 con 0x7f8a7c1067d0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.611+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a7c0fd100 con 0x7f8a7c1067d0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.612+0000 7f8a72ffd640 1 -- 172.21.15.196:0/1715783383 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8a740051d0 con 0x7f8a7c1067d0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.612+0000 7f8a72ffd640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8a680a3240 0x7f8a680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.613+0000 7f8a81db5640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8a680a3240 0x7f8a680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.613+0000 7f8a72ffd640 1 -- 172.21.15.196:0/1715783383 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5430+0+0 (secure 0 0 0) 0x7f8a740cee90 con 0x7f8a7c1067d0 2024-09-19T04:02:05.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.613+0000 7f8a81db5640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8a680a3240 0x7f8a680a5700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a78000c00 tx=0x7f8a78031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:02:05.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.617+0000 7f8a72ffd640 1 -- 172.21.15.196:0/1715783383 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8a74014030 con 0x7f8a7c1067d0 2024-09-19T04:02:05.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:05.792+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8a7c062630 con 0x7f8a680a3240 2024-09-19T04:02:05.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:05.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T04:02:05.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:05.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:05.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='osd.7 ' entity='osd.7' 2024-09-19T04:02:05.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi111 2024-09-19T04:02:05.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] boot 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:02:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='osd.7 ' entity='osd.7' 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi111 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] boot 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:02:05.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:02:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T04:02:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='osd.7 ' entity='osd.7' 2024-09-19T04:02:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi111 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: osd.7 [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] boot 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:02:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:02:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:06 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:02:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:06 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:02:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:06 smithi196 ceph-mon[35731]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:06 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:06 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:06.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:06 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:06.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:06 smithi196 ceph-mon[35731]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:02:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:06 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:02:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:06 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:02:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:06 smithi111 ceph-mon[35365]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:06 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:06 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:06 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:06 smithi111 ceph-mon[35365]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:02:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:06 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:02:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:06 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:02:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:06 smithi019 ceph-mon[30938]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:06 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:06 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:06 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:06.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:06 smithi019 ceph-mon[30938]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:02:07.901 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:07 smithi196 ceph-mon[35731]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:07.901 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:07 smithi196 ceph-mon[35731]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:02:07.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:07 smithi111 ceph-mon[35365]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:07.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:07 smithi111 ceph-mon[35365]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:02:07.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:07 smithi019 ceph-mon[30938]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:07.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:07 smithi019 ceph-mon[30938]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:02:08.800 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:08 smithi196 ceph-mon[35731]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T04:02:08.800 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:08 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/370525325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]: dispatch 2024-09-19T04:02:08.800 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:08 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]: dispatch 2024-09-19T04:02:08.800 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:08 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]': finished 2024-09-19T04:02:08.800 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:08 smithi196 ceph-mon[35731]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:02:08.800 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:08 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:08.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:08 smithi111 ceph-mon[35365]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T04:02:08.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:08 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/370525325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]: dispatch 2024-09-19T04:02:08.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:08 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]: dispatch 2024-09-19T04:02:08.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:08 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]': finished 2024-09-19T04:02:08.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:08 smithi111 ceph-mon[35365]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:02:08.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:08 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:08.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:08 smithi019 ceph-mon[30938]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T04:02:08.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:08 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/370525325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]: dispatch 2024-09-19T04:02:08.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:08 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]: dispatch 2024-09-19T04:02:08.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:08 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cdff67f-fc25-43ca-99e0-cb2063ee063c"}]': finished 2024-09-19T04:02:08.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:08 smithi019 ceph-mon[30938]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T04:02:08.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:08 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:09.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:09 smithi111 ceph-mon[35365]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:09.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:09 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/2744049687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:02:09.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:09 smithi019 ceph-mon[30938]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:09.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:09 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/2744049687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:02:09.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:09 smithi196 ceph-mon[35731]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:09.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:09 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/2744049687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:02:11.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:11 smithi111 ceph-mon[35365]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:11.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:11 smithi019 ceph-mon[30938]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:11.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:11 smithi196 ceph-mon[35731]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:13.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:13 smithi196 ceph-mon[35731]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:13.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:13 smithi111 ceph-mon[35365]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:13.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:13 smithi019 ceph-mon[30938]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:15.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:15 smithi196 ceph-mon[35731]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:15.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:15 smithi111 ceph-mon[35365]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:15.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:15 smithi019 ceph-mon[30938]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:16.774 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:02:16.774 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:16.774 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:16 smithi196 ceph-mon[35731]: Deploying daemon osd.8 on smithi196 2024-09-19T04:02:16.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:02:16.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:16.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:16 smithi111 ceph-mon[35365]: Deploying daemon osd.8 on smithi196 2024-09-19T04:02:16.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:02:16.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:16.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:16 smithi019 ceph-mon[30938]: Deploying daemon osd.8 on smithi196 2024-09-19T04:02:17.810 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:17 smithi196 ceph-mon[35731]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:17.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:17 smithi111 ceph-mon[35365]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:17.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:17 smithi019 ceph-mon[30938]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:19.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:19 smithi196 ceph-mon[35731]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:19.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:19 smithi111 ceph-mon[35365]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:19.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:19 smithi019 ceph-mon[30938]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:21.651 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:21 smithi196 ceph-mon[35731]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:21 smithi111 ceph-mon[35365]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:21.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:21 smithi019 ceph-mon[30938]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:22.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:22.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:22.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:22.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:22.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:22.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:24.350 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:24 smithi196 ceph-mon[35731]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:24.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:24 smithi111 ceph-mon[35365]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:24.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:24 smithi019 ceph-mon[30938]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:26.080 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.078+0000 7f8a72ffd640 1 -- 172.21.15.196:0/1715783383 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8a7c062630 con 0x7f8a680a3240 2024-09-19T04:02:26.080 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-09-19T04:02:26.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8a680a3240 msgr2=0x7f8a680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:26.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8a680a3240 0x7f8a680a5700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a78000c00 tx=0x7f8a78031400 comp rx=0 tx=0).stop 2024-09-19T04:02:26.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1067d0 msgr2=0x7f8a7c19f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:26.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1067d0 0x7f8a7c19f2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400cad0 tx=0x7f8a7400cf90 comp rx=0 tx=0).stop 2024-09-19T04:02:26.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 shutdown_connections 2024-09-19T04:02:26.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8a680a3240 0x7f8a680a5700 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:26.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a7c1067d0 0x7f8a7c19f2c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:26.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a7c103d20 0x7f8a7c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:26.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 --2- 172.21.15.196:0/1715783383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a7c1013f0 0x7f8a7c100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:26.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.080+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 >> 172.21.15.196:0/1715783383 conn(0x7f8a7c0fb610 msgr2=0x7f8a7c101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:02:26.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.081+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 shutdown_connections 2024-09-19T04:02:26.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:02:26.081+0000 7f8a84019640 1 -- 172.21.15.196:0/1715783383 wait complete. 2024-09-19T04:02:26.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:26 smithi111 ceph-mon[35365]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:26.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:26.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:26.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:26 smithi019 ceph-mon[30938]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:26.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:26.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:26.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:26 smithi196 ceph-mon[35731]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:26.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:26.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:26.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:26.754 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T04:02:26.757 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-09-19T04:02:26.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:02:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:02:27.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:27 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:27.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:27 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:27.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:27 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:27.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:27 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:27.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:27 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:27.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:27 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:27.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:27 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:27.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:27 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:27.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:27 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:28.154 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:28 smithi196 ceph-mon[35731]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:28.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:28 smithi019 ceph-mon[30938]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:28.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:28 smithi111 ceph-mon[35365]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:28.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:02:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:02:28.313+0000 7f301653f640 -1 osd.8 0 log_to_monitors true 2024-09-19T04:02:29.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:29 smithi196 ceph-mon[35731]: from='osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:02:29.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:29 smithi196 ceph-mon[35731]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:02:29.333 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:29 smithi019 ceph-mon[30938]: from='osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:02:29.333 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:29 smithi019 ceph-mon[30938]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:02:29.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:29 smithi111 ceph-mon[35365]: from='osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:02:29.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:29 smithi111 ceph-mon[35365]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T04:02:29.879 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:02:29.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-09-19T04:02:30.163 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:02:30.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:30.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:02:30.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T04:02:30.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:30.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:02:30.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:02:30.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:30.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:30.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:30.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:02:30.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T04:02:30.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:30.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T04:02:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:30.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 -- 172.21.15.19:0/952074524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6294101d10 msgr2=0x7f6294102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 --2- 172.21.15.19:0/952074524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6294101d10 0x7f6294102190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6284009900 tx=0x7f628402a410 comp rx=0 tx=0).stop 2024-09-19T04:02:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 -- 172.21.15.19:0/952074524 shutdown_connections 2024-09-19T04:02:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 --2- 172.21.15.19:0/952074524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62941026d0 0x7f6294108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 --2- 172.21.15.19:0/952074524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6294101d10 0x7f6294102190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 --2- 172.21.15.19:0/952074524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6294100b10 0x7f6294100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 -- 172.21.15.19:0/952074524 >> 172.21.15.19:0/952074524 conn(0x7f62940fc2a0 msgr2=0x7f62940fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:02:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 -- 172.21.15.19:0/952074524 shutdown_connections 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.658+0000 7f629b6f9640 1 -- 172.21.15.19:0/952074524 wait complete. 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f629b6f9640 1 Processor -- start 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f629b6f9640 1 -- start start 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f629b6f9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6294100b10 0x7f6294197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f629b6f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6294101d10 0x7f6294198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f629b6f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62941026d0 0x7f629419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f629b6f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f629410c790 con 0x7f62941026d0 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f629b6f9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f629410c610 con 0x7f6294100b10 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f629b6f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f629410c910 con 0x7f6294101d10 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299c96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62941026d0 0x7f629419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299c96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62941026d0 0x7f629419f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49270/0 (socket says 172.21.15.19:49270) 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299c96640 1 -- 172.21.15.19:0/1417886816 learned_addr learned my addr 172.21.15.19:0/1417886816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299495640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6294100b10 0x7f6294197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6298c94640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6294101d10 0x7f6294198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299c96640 1 -- 172.21.15.19:0/1417886816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6294101d10 msgr2=0x7f6294198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299c96640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6294101d10 0x7f6294198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299c96640 1 -- 172.21.15.19:0/1417886816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6294100b10 msgr2=0x7f6294197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299c96640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6294100b10 0x7f6294197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6299c96640 1 -- 172.21.15.19:0/1417886816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629419fb70 con 0x7f62941026d0 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.659+0000 7f6298c94640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6294101d10 0x7f6294198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.660+0000 7f6299c96640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62941026d0 0x7f629419f500 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f628c009fd0 tx=0x7f628c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.660+0000 7f628a7fc640 1 -- 172.21.15.19:0/1417886816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c017070 con 0x7f62941026d0 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.660+0000 7f628a7fc640 1 -- 172.21.15.19:0/1417886816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f628c003e50 con 0x7f62941026d0 2024-09-19T04:02:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.660+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629419fe60 con 0x7f62941026d0 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.660+0000 7f628a7fc640 1 -- 172.21.15.19:0/1417886816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c0106c0 con 0x7f62941026d0 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.660+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62941a02c0 con 0x7f62941026d0 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.661+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62941a04f0 con 0x7f62941026d0 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.662+0000 7f628a7fc640 1 -- 172.21.15.19:0/1417886816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f628c003970 con 0x7f62941026d0 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.663+0000 7f628a7fc640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62800a3270 0x7f62800a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.663+0000 7f6299495640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62800a3270 0x7f62800a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.663+0000 7f628a7fc640 1 -- 172.21.15.19:0/1417886816 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5549+0+0 (secure 0 0 0) 0x7f628c0cecf0 con 0x7f62941026d0 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.663+0000 7f6299495640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62800a3270 0x7f62800a5730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6294101b70 tx=0x7f6290005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:02:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.667+0000 7f628a7fc640 1 -- 172.21.15.19:0/1417886816 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f628c09c440 con 0x7f62941026d0 2024-09-19T04:02:30.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:30.857+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f62940fd100 con 0x7f62800a3270 2024-09-19T04:02:30.958 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:02:30.486+0000 7f30124c2640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T04:02:30.958 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:02:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:02:30.496+0000 7f300a2e2640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:02:31.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi196 2024-09-19T04:02:31.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-19T04:02:31.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:02:31.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T04:02:31.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:31.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:31.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: from='osd.8 ' entity='osd.8' 2024-09-19T04:02:31.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:31.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:31.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi196 2024-09-19T04:02:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-19T04:02:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:02:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T04:02:31.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:31.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:31.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: from='osd.8 ' entity='osd.8' 2024-09-19T04:02:31.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:31.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:31.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi196 2024-09-19T04:02:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-19T04:02:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:02:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T04:02:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: from='osd.8 ' entity='osd.8' 2024-09-19T04:02:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:31.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:32 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:02:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:32 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:02:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:32 smithi111 ceph-mon[35365]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:32 smithi111 ceph-mon[35365]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:32 smithi111 ceph-mon[35365]: osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] boot 2024-09-19T04:02:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:32 smithi111 ceph-mon[35365]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:02:32.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:32.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:32 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:02:32.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:32 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:02:32.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:32 smithi019 ceph-mon[30938]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:32.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:32 smithi019 ceph-mon[30938]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:32.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:32.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:32 smithi019 ceph-mon[30938]: osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] boot 2024-09-19T04:02:32.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:32 smithi019 ceph-mon[30938]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:02:32.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:32 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:02:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:32 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:02:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:32 smithi196 ceph-mon[35731]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:02:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:32 smithi196 ceph-mon[35731]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:32 smithi196 ceph-mon[35731]: osd.8 [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] boot 2024-09-19T04:02:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:32 smithi196 ceph-mon[35731]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T04:02:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:02:33.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:33 smithi111 ceph-mon[35365]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:02:33.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:33 smithi019 ceph-mon[30938]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:02:33.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:33 smithi196 ceph-mon[35731]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T04:02:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:34 smithi111 ceph-mon[35365]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:34 smithi111 ceph-mon[35365]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T04:02:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:34 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3708995258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f507340-9ceb-4005-9247-bb7fac27bdf3"}]: dispatch 2024-09-19T04:02:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:34 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3708995258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f507340-9ceb-4005-9247-bb7fac27bdf3"}]': finished 2024-09-19T04:02:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:34 smithi111 ceph-mon[35365]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:02:34.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:34.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:34 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/992693151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:02:34.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:34 smithi019 ceph-mon[30938]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:34.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:34 smithi019 ceph-mon[30938]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T04:02:34.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:34 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3708995258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f507340-9ceb-4005-9247-bb7fac27bdf3"}]: dispatch 2024-09-19T04:02:34.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:34 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3708995258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f507340-9ceb-4005-9247-bb7fac27bdf3"}]': finished 2024-09-19T04:02:34.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:34 smithi019 ceph-mon[30938]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:02:34.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:34.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:34 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/992693151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:02:34.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:34 smithi196 ceph-mon[35731]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:34.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:34 smithi196 ceph-mon[35731]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T04:02:34.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:34 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3708995258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f507340-9ceb-4005-9247-bb7fac27bdf3"}]: dispatch 2024-09-19T04:02:34.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:34 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3708995258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f507340-9ceb-4005-9247-bb7fac27bdf3"}]': finished 2024-09-19T04:02:34.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:34 smithi196 ceph-mon[35731]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T04:02:34.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:34.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:34 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/992693151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:02:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:35 smithi111 ceph-mon[35365]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T04:02:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:35 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:35 smithi019 ceph-mon[30938]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T04:02:35.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:35 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:35 smithi196 ceph-mon[35731]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T04:02:35.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:35 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:36.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:36 smithi111 ceph-mon[35365]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:36.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:36 smithi111 ceph-mon[35365]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T04:02:36.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:36 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:36.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:36 smithi019 ceph-mon[30938]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:36.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:36 smithi019 ceph-mon[30938]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T04:02:36.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:36 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:36.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:36 smithi196 ceph-mon[35731]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:36.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:36 smithi196 ceph-mon[35731]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T04:02:36.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:36 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:37 smithi111 ceph-mon[35365]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:02:37.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:37 smithi019 ceph-mon[30938]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:02:37.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:37 smithi196 ceph-mon[35731]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:02:39.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:39 smithi196 ceph-mon[35731]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-19T04:02:39.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:39 smithi019 ceph-mon[30938]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-19T04:02:39.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:39 smithi111 ceph-mon[35365]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-19T04:02:41.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:41 smithi019 ceph-mon[30938]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T04:02:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:41 smithi196 ceph-mon[35731]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T04:02:41.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:41 smithi111 ceph-mon[35365]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T04:02:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:02:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:42 smithi196 ceph-mon[35731]: Deploying daemon osd.9 on smithi019 2024-09-19T04:02:42.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:02:42.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:42.784 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:42 smithi019 ceph-mon[30938]: Deploying daemon osd.9 on smithi019 2024-09-19T04:02:42.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T04:02:42.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:42.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:42 smithi111 ceph-mon[35365]: Deploying daemon osd.9 on smithi019 2024-09-19T04:02:43.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:43 smithi196 ceph-mon[35731]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:02:43.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:43 smithi019 ceph-mon[30938]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:02:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:43 smithi111 ceph-mon[35365]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:02:45.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:45 smithi019 ceph-mon[30938]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:02:45.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:45 smithi196 ceph-mon[35731]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:02:45.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:45 smithi111 ceph-mon[35365]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T04:02:47.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:47 smithi196 ceph-mon[35731]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:02:47.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:47 smithi019 ceph-mon[30938]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:02:47.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:47 smithi111 ceph-mon[35365]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:02:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:48.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:48.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:48.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:50.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:50 smithi019 ceph-mon[30938]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:50.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:50 smithi196 ceph-mon[35731]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:50 smithi111 ceph-mon[35365]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:51.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.850+0000 7f628a7fc640 1 -- 172.21.15.19:0/1417886816 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f62940fd100 con 0x7f62800a3270 2024-09-19T04:02:51.851 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-09-19T04:02:51.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62800a3270 msgr2=0x7f62800a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62800a3270 0x7f62800a5730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6294101b70 tx=0x7f6290005f90 comp rx=0 tx=0).stop 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62941026d0 msgr2=0x7f629419f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62941026d0 0x7f629419f500 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f628c009fd0 tx=0x7f628c00b260 comp rx=0 tx=0).stop 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 shutdown_connections 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62800a3270 0x7f62800a5730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62941026d0 0x7f629419f500 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6294101d10 0x7f6294198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 --2- 172.21.15.19:0/1417886816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6294100b10 0x7f6294197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 >> 172.21.15.19:0/1417886816 conn(0x7f62940fc2a0 msgr2=0x7f62940fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 shutdown_connections 2024-09-19T04:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:02:51.852+0000 7f629b6f9640 1 -- 172.21.15.19:0/1417886816 wait complete. 2024-09-19T04:02:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:52.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:52.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:52.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:52 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:52 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:52.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:02:52.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:52 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:52.572 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T04:02:52.574 INFO:tasks.cephadm:Deploying osd.10 on smithi111 with /dev/vg_nvme/lv_1... 2024-09-19T04:02:52.575 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:02:52.827 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:02:54.256 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:54 smithi111 ceph-mon[35365]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:54.278 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:54 smithi019 ceph-mon[30938]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:54.279 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:02:54 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:02:54.225+0000 7fb69e5a0640 -1 osd.9 0 log_to_monitors true 2024-09-19T04:02:54.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:54 smithi196 ceph-mon[35731]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:55.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:55 smithi111 ceph-mon[35365]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:02:55.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:55 smithi019 ceph-mon[30938]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:02:55.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:55 smithi196 ceph-mon[35731]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T04:02:55.586 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-19T04:02:55.614 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2024-09-19T04:02:55.875 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:02:56.154 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:56.154 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:02:56.154 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T04:02:56.154 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:02:56.154 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:56.154 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.154 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.154 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:56.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:56.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 -- 172.21.15.111:0/2827572478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124104ff0 msgr2=0x7f51241073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:56.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 --2- 172.21.15.111:0/2827572478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124104ff0 0x7f51241073e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f510c009900 tx=0x7f510c02a410 comp rx=0 tx=0).stop 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 -- 172.21.15.111:0/2827572478 shutdown_connections 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 --2- 172.21.15.111:0/2827572478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5124107920 0x7f5124109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 --2- 172.21.15.111:0/2827572478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124104ff0 0x7f51241073e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 --2- 172.21.15.111:0/2827572478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51241026c0 0x7f5124104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 -- 172.21.15.111:0/2827572478 >> 172.21.15.111:0/2827572478 conn(0x7f51240fc280 msgr2=0x7f51240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 -- 172.21.15.111:0/2827572478 shutdown_connections 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.381+0000 7f5129941640 1 -- 172.21.15.111:0/2827572478 wait complete. 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5129941640 1 Processor -- start 2024-09-19T04:02:56.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5129941640 1 -- start start 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5129941640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51241026c0 0x7f5124197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5129941640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5124104ff0 0x7f51241984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5129941640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124107920 0x7f512419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5129941640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f512410d8d0 con 0x7f5124104ff0 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5129941640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f512410d750 con 0x7f5124107920 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5129941640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f512410da50 con 0x7f51241026c0 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f51237fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124107920 0x7f512419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f51237fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124107920 0x7f512419f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36090/0 (socket says 172.21.15.111:36090) 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f51237fe640 1 -- 172.21.15.111:0/4155206206 learned_addr learned my addr 172.21.15.111:0/4155206206 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.382+0000 7f5122ffd640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51241026c0 0x7f5124197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f51227fc640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5124104ff0 0x7f51241984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:56.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f51237fe640 1 -- 172.21.15.111:0/4155206206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51241026c0 msgr2=0x7f5124197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f51237fe640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51241026c0 0x7f5124197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f51237fe640 1 -- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5124104ff0 msgr2=0x7f51241984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f51237fe640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5124104ff0 0x7f51241984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f51237fe640 1 -- 172.21.15.111:0/4155206206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512419fbe0 con 0x7f5124107920 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f51237fe640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124107920 0x7f512419f570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5114009fd0 tx=0x7f511400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f512893f640 1 -- 172.21.15.111:0/4155206206 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114011830 con 0x7f5124107920 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512419fed0 con 0x7f5124107920 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f512893f640 1 -- 172.21.15.111:0/4155206206 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5114016070 con 0x7f5124107920 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f512893f640 1 -- 172.21.15.111:0/4155206206 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511400f2e0 con 0x7f5124107920 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.383+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241a03b0 con 0x7f5124107920 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.384+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51240fdd70 con 0x7f5124107920 2024-09-19T04:02:56.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.385+0000 7f512893f640 1 -- 172.21.15.111:0/4155206206 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f51140119d0 con 0x7f5124107920 2024-09-19T04:02:56.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.386+0000 7f512893f640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51100a3270 0x7f51100a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:02:56.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.386+0000 7f512893f640 1 -- 172.21.15.111:0/4155206206 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5981+0+0 (secure 0 0 0) 0x7f51140cedb0 con 0x7f5124107920 2024-09-19T04:02:56.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.386+0000 7f5122ffd640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51100a3270 0x7f51100a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:02:56.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.387+0000 7f5122ffd640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51100a3270 0x7f51100a5730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f511c005fd0 tx=0x7f511c009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:02:56.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.389+0000 7f512893f640 1 -- 172.21.15.111:0/4155206206 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f511409c3f0 con 0x7f5124107920 2024-09-19T04:02:56.431 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:02:56.035+0000 7fb69a523640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T04:02:56.431 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:02:56.049+0000 7fb69634b640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:02:56.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:56.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:02:56.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T04:02:56.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:56.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T04:02:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T04:02:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-19T04:02:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:02:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:02:56.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:02:56.573+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5124062630 con 0x7f51100a3270 2024-09-19T04:02:57.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi019 2024-09-19T04:02:57.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-19T04:02:57.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:02:57.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T04:02:57.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:57.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:57.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:57.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:57.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:57.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:57 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:57.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi019 2024-09-19T04:02:57.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-19T04:02:57.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:02:57.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T04:02:57.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:57.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:57.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:57.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:57.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:57.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:57 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:57.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi019 2024-09-19T04:02:57.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-19T04:02:57.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: from='osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-19T04:02:57.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T04:02:57.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:57.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:57.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:02:57.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:02:57.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:02:57.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:57 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:58.340 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:58 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:02:58.340 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:58 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:02:58.341 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:58 smithi111 ceph-mon[35365]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:58.341 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:58 smithi111 ceph-mon[35365]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:58.341 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:58 smithi111 ceph-mon[35365]: osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] boot 2024-09-19T04:02:58.341 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:58 smithi111 ceph-mon[35365]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T04:02:58.341 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:58 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:58 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:02:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:58 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:02:58.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:58 smithi019 ceph-mon[30938]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:58.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:58 smithi019 ceph-mon[30938]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:58.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:58 smithi019 ceph-mon[30938]: osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] boot 2024-09-19T04:02:58.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:58 smithi019 ceph-mon[30938]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T04:02:58.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:58 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:58.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:58 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:02:58.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:58 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:02:58.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:58 smithi196 ceph-mon[35731]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T04:02:58.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:58 smithi196 ceph-mon[35731]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:02:58.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:58 smithi196 ceph-mon[35731]: osd.9 [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] boot 2024-09-19T04:02:58.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:58 smithi196 ceph-mon[35731]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T04:02:58.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:58 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T04:02:59.342 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:02:59 smithi111 ceph-mon[35365]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T04:02:59.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:02:59 smithi019 ceph-mon[30938]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T04:02:59.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:02:59 smithi196 ceph-mon[35731]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T04:03:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:00 smithi111 ceph-mon[35365]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:00 smithi111 ceph-mon[35365]: from='client.? 172.21.15.111:0/4292455655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]: dispatch 2024-09-19T04:03:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:00 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]: dispatch 2024-09-19T04:03:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:00 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]': finished 2024-09-19T04:03:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:00 smithi111 ceph-mon[35365]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:03:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:00 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:00.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:00 smithi111 ceph-mon[35365]: from='client.? 172.21.15.111:0/783582538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:03:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:00 smithi019 ceph-mon[30938]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:00 smithi019 ceph-mon[30938]: from='client.? 172.21.15.111:0/4292455655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]: dispatch 2024-09-19T04:03:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:00 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]: dispatch 2024-09-19T04:03:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:00 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]': finished 2024-09-19T04:03:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:00 smithi019 ceph-mon[30938]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:03:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:00 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:00 smithi019 ceph-mon[30938]: from='client.? 172.21.15.111:0/783582538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:03:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:00 smithi196 ceph-mon[35731]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:00 smithi196 ceph-mon[35731]: from='client.? 172.21.15.111:0/4292455655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]: dispatch 2024-09-19T04:03:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:00 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]: dispatch 2024-09-19T04:03:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:00 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17727a7c-08c8-46ba-b9f9-e0704680e6ff"}]': finished 2024-09-19T04:03:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:00 smithi196 ceph-mon[35731]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T04:03:00.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:00 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:00.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:00 smithi196 ceph-mon[35731]: from='client.? 172.21.15.111:0/783582538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:03:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:02 smithi111 ceph-mon[35365]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:02.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:02 smithi019 ceph-mon[30938]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:02.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:02 smithi196 ceph-mon[35731]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:04.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:04 smithi111 ceph-mon[35365]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:04.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:04 smithi019 ceph-mon[30938]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:04 smithi196 ceph-mon[35731]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:06.368 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:06 smithi111 ceph-mon[35365]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:06.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:06 smithi019 ceph-mon[30938]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:06.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:06 smithi196 ceph-mon[35731]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:08.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:08 smithi111 ceph-mon[35365]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:08.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:08 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:03:08.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:08 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:08.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:08 smithi019 ceph-mon[30938]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:08 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:03:08.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:08 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:08 smithi196 ceph-mon[35731]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:08 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T04:03:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:08 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:09.364 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:09 smithi111 ceph-mon[35365]: Deploying daemon osd.10 on smithi111 2024-09-19T04:03:09.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:09 smithi019 ceph-mon[30938]: Deploying daemon osd.10 on smithi111 2024-09-19T04:03:09.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:09 smithi196 ceph-mon[35731]: Deploying daemon osd.10 on smithi111 2024-09-19T04:03:10.372 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:10 smithi111 ceph-mon[35365]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:10 smithi019 ceph-mon[30938]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:10 smithi196 ceph-mon[35731]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:12.361 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:12 smithi111 ceph-mon[35365]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:12.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:12 smithi019 ceph-mon[30938]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:12.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:12 smithi196 ceph-mon[35731]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:14 smithi111 ceph-mon[35365]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:14 smithi019 ceph-mon[30938]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:14 smithi196 ceph-mon[35731]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:16.349 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:16 smithi111 ceph-mon[35365]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:16 smithi019 ceph-mon[30938]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:16.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:16 smithi196 ceph-mon[35731]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:17.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.646+0000 7f512893f640 1 -- 172.21.15.111:0/4155206206 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5124062630 con 0x7f51100a3270 2024-09-19T04:03:17.649 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 10 on host 'smithi111' 2024-09-19T04:03:17.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51100a3270 msgr2=0x7f51100a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:17.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51100a3270 0x7f51100a5730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f511c005fd0 tx=0x7f511c009870 comp rx=0 tx=0).stop 2024-09-19T04:03:17.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124107920 msgr2=0x7f512419f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:17.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124107920 0x7f512419f570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5114009fd0 tx=0x7f511400b8e0 comp rx=0 tx=0).stop 2024-09-19T04:03:17.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 shutdown_connections 2024-09-19T04:03:17.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51100a3270 0x7f51100a5730 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:17.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5124107920 0x7f512419f570 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:17.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5124104ff0 0x7f51241984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:17.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 --2- 172.21.15.111:0/4155206206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51241026c0 0x7f5124197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:17.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 >> 172.21.15.111:0/4155206206 conn(0x7f51240fc280 msgr2=0x7f5124103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:17.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 shutdown_connections 2024-09-19T04:03:17.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-19T04:03:17.649+0000 7f5129941640 1 -- 172.21.15.111:0/4155206206 wait complete. 2024-09-19T04:03:17.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:17.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:17.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:17.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:17 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:18.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:17 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:18.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:18.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:18.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:18.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:17 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:18.253 DEBUG:teuthology.orchestra.run.smithi111:osd.10> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T04:03:18.255 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-09-19T04:03:18.255 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T04:03:18.509 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:03:19.942 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:19 smithi196 ceph-mon[35731]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:20.139 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:19 smithi111 ceph-mon[35365]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:20.139 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:03:20 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:03:20.004+0000 7f0ade79a640 -1 osd.10 0 log_to_monitors true 2024-09-19T04:03:20.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:19 smithi019 ceph-mon[30938]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:21.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:20 smithi111 ceph-mon[35365]: from='osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:03:21.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:20 smithi111 ceph-mon[35365]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:03:21.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:20 smithi019 ceph-mon[30938]: from='osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:03:21.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:20 smithi019 ceph-mon[30938]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:03:21.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:20 smithi196 ceph-mon[35731]: from='osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:03:21.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:20 smithi196 ceph-mon[35731]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T04:03:21.324 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T04:03:21.350 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-09-19T04:03:21.601 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:03:22.024 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.025 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.026 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.026 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.026 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:22.026 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:22.026 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 -- 172.21.15.196:0/550156625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 msgr2=0x7f721c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 --2- 172.21.15.196:0/550156625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 0x7f721c102150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7208009900 tx=0x7f720802a410 comp rx=0 tx=0).stop 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 -- 172.21.15.196:0/550156625 shutdown_connections 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 --2- 172.21.15.196:0/550156625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c102690 0x7f721c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 --2- 172.21.15.196:0/550156625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 0x7f721c102150 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 --2- 172.21.15.196:0/550156625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f721c100ad0 0x7f721c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 -- 172.21.15.196:0/550156625 >> 172.21.15.196:0/550156625 conn(0x7f721c0fc280 msgr2=0x7f721c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 -- 172.21.15.196:0/550156625 shutdown_connections 2024-09-19T04:03:22.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 -- 172.21.15.196:0/550156625 wait complete. 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 Processor -- start 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.107+0000 7f7222e71640 1 -- start start 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7222e71640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f721c100ad0 0x7f721c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7222e71640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 0x7f721c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7222e71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c102690 0x7f721c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7222e71640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10d8e0 con 0x7f721c102690 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7222e71640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10d760 con 0x7f721c100ad0 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7222e71640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10da60 con 0x7f721c101cd0 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 0x7f721c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7220c0d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f721c100ad0 0x7f721c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:22.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 0x7f721c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58346/0 (socket says 172.21.15.196:58346) 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 -- 172.21.15.196:0/2866469443 learned_addr learned my addr 172.21.15.196:0/2866469443 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f722140e640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c102690 0x7f721c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 -- 172.21.15.196:0/2866469443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f721c100ad0 msgr2=0x7f721c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f721c100ad0 0x7f721c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 -- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c102690 msgr2=0x7f721c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c102690 0x7f721c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 -- 172.21.15.196:0/2866469443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c19fb90 con 0x7f721c101cd0 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7220c0d640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f721c100ad0 0x7f721c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f721bfff640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 0x7f721c1984a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7208009230 tx=0x7f7208004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.108+0000 7f7219ffb640 1 -- 172.21.15.196:0/2866469443 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720803f030 con 0x7f721c101cd0 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.109+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c19fe20 con 0x7f721c101cd0 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.109+0000 7f7219ffb640 1 -- 172.21.15.196:0/2866469443 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7208003ce0 con 0x7f721c101cd0 2024-09-19T04:03:22.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.109+0000 7f7219ffb640 1 -- 172.21.15.196:0/2866469443 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208035070 con 0x7f721c101cd0 2024-09-19T04:03:22.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.109+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c1a0300 con 0x7f721c101cd0 2024-09-19T04:03:22.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.110+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721c107e70 con 0x7f721c101cd0 2024-09-19T04:03:22.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.111+0000 7f7219ffb640 1 -- 172.21.15.196:0/2866469443 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7208042030 con 0x7f721c101cd0 2024-09-19T04:03:22.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.112+0000 7f7219ffb640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f720c0a3270 0x7f720c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:22.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.112+0000 7f7219ffb640 1 -- 172.21.15.196:0/2866469443 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6414+0+0 (secure 0 0 0) 0x7f72080efbd0 con 0x7f721c101cd0 2024-09-19T04:03:22.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.112+0000 7f7220c0d640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f720c0a3270 0x7f720c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:22.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.113+0000 7f7220c0d640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f720c0a3270 0x7f720c0a5730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7210005fd0 tx=0x7f721000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:22.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.116+0000 7f7219ffb640 1 -- 172.21.15.196:0/2866469443 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f72080bc970 con 0x7f721c101cd0 2024-09-19T04:03:22.155 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:03:21.867+0000 7f0ada71d640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T04:03:22.155 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:03:21.879+0000 7f0ad6545640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:03:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:03:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T04:03:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:22.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T04:03:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:22.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:22.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:22.293+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f721c062630 con 0x7f720c0a3270 2024-09-19T04:03:22.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi111 2024-09-19T04:03:22.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-19T04:03:22.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] boot 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T04:03:22.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:23.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi111 2024-09-19T04:03:23.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-19T04:03:23.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:03:23.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T04:03:23.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:23.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:23.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:03:23.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:03:23.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:03:23.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:23.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] boot 2024-09-19T04:03:23.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T04:03:23.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:23.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi111 2024-09-19T04:03:23.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-19T04:03:23.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-19T04:03:23.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T04:03:23.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:23.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:23.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:03:23.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:03:23.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:03:23.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:23.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: osd.10 [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] boot 2024-09-19T04:03:23.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T04:03:23.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T04:03:24.143 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:23 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:03:24.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:23 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:03:24.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:23 smithi196 ceph-mon[35731]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:24.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:23 smithi196 ceph-mon[35731]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T04:03:24.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:23 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:03:24.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:23 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:03:24.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:23 smithi111 ceph-mon[35365]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:24.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:23 smithi111 ceph-mon[35365]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T04:03:24.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:23 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:03:24.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:23 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:03:24.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:23 smithi019 ceph-mon[30938]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T04:03:24.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:23 smithi019 ceph-mon[30938]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T04:03:25.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:25 smithi196 ceph-mon[35731]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:25.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:25 smithi196 ceph-mon[35731]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T04:03:25.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:25 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/4037636420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]: dispatch 2024-09-19T04:03:25.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:25 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]: dispatch 2024-09-19T04:03:25.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:25 smithi196 ceph-mon[35731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]': finished 2024-09-19T04:03:25.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:25 smithi196 ceph-mon[35731]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T04:03:25.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:25 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:25.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:25 smithi196 ceph-mon[35731]: from='client.? 172.21.15.196:0/1931547494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:03:25.907 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:25 smithi111 ceph-mon[35365]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:25.908 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:25 smithi111 ceph-mon[35365]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T04:03:25.908 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:25 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/4037636420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]: dispatch 2024-09-19T04:03:25.908 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:25 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]: dispatch 2024-09-19T04:03:25.909 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:25 smithi111 ceph-mon[35365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]': finished 2024-09-19T04:03:25.910 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:25 smithi111 ceph-mon[35365]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T04:03:25.910 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:25 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:25.911 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:25 smithi111 ceph-mon[35365]: from='client.? 172.21.15.196:0/1931547494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:03:25.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:25 smithi019 ceph-mon[30938]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:25.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:25 smithi019 ceph-mon[30938]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T04:03:25.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:25 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/4037636420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]: dispatch 2024-09-19T04:03:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:25 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]: dispatch 2024-09-19T04:03:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:25 smithi019 ceph-mon[30938]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f71692d8-81b5-439b-b1c2-8972209815cb"}]': finished 2024-09-19T04:03:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:25 smithi019 ceph-mon[30938]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T04:03:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:25 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:25.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:25 smithi019 ceph-mon[30938]: from='client.? 172.21.15.196:0/1931547494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:03:27.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:26 smithi111 ceph-mon[35365]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:03:27.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:26 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:27.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:26 smithi019 ceph-mon[30938]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:03:27.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:26 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:27.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:26 smithi196 ceph-mon[35731]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T04:03:27.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:26 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:28.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:27 smithi111 ceph-mon[35365]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-19T04:03:28.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:27 smithi019 ceph-mon[30938]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-19T04:03:28.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:27 smithi196 ceph-mon[35731]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-19T04:03:30.125 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:29 smithi196 ceph-mon[35731]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-19T04:03:30.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:29 smithi111 ceph-mon[35365]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-19T04:03:30.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:29 smithi019 ceph-mon[30938]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-19T04:03:32.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:31 smithi111 ceph-mon[35365]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:03:32.157 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:31 smithi196 ceph-mon[35731]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:03:32.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:31 smithi019 ceph-mon[30938]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T04:03:34.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:33 smithi196 ceph-mon[35731]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:03:34.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:03:34.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:33 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:34.012 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:33 smithi196 ceph-mon[35731]: Deploying daemon osd.11 on smithi196 2024-09-19T04:03:34.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:33 smithi111 ceph-mon[35365]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:03:34.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:03:34.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:33 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:34.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:33 smithi111 ceph-mon[35365]: Deploying daemon osd.11 on smithi196 2024-09-19T04:03:34.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:33 smithi019 ceph-mon[30938]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T04:03:34.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T04:03:34.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:33 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:34.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:33 smithi019 ceph-mon[30938]: Deploying daemon osd.11 on smithi196 2024-09-19T04:03:36.140 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:35 smithi196 ceph-mon[35731]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T04:03:36.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:35 smithi111 ceph-mon[35365]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T04:03:36.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:35 smithi019 ceph-mon[30938]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T04:03:38.096 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:37 smithi196 ceph-mon[35731]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:38.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:37 smithi111 ceph-mon[35365]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:38.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:37 smithi019 ceph-mon[30938]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:40.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:39 smithi111 ceph-mon[35365]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:40.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:40.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:40.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:39 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:40.165 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:39 smithi196 ceph-mon[35731]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:40.165 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:40.165 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:40.165 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:39 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:40.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:39 smithi019 ceph-mon[30938]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:40.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:40.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:40.185 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:39 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:42.135 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:41 smithi196 ceph-mon[35731]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:41 smithi019 ceph-mon[30938]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:42.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:41 smithi111 ceph-mon[35365]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:43.055 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-09-19T04:03:43.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.054+0000 7f7219ffb640 1 -- 172.21.15.196:0/2866469443 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f721c062630 con 0x7f720c0a3270 2024-09-19T04:03:43.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.056+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f720c0a3270 msgr2=0x7f720c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:43.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.056+0000 7f7222e71640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f720c0a3270 0x7f720c0a5730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7210005fd0 tx=0x7f721000b390 comp rx=0 tx=0).stop 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 msgr2=0x7f721c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 0x7f721c1984a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7208009230 tx=0x7f7208004360 comp rx=0 tx=0).stop 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 shutdown_connections 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f720c0a3270 0x7f720c0a5730 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c102690 0x7f721c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101cd0 0x7f721c1984a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 --2- 172.21.15.196:0/2866469443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f721c100ad0 0x7f721c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 >> 172.21.15.196:0/2866469443 conn(0x7f721c0fc280 msgr2=0x7f721c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 shutdown_connections 2024-09-19T04:03:43.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:43.057+0000 7f7222e71640 1 -- 172.21.15.196:0/2866469443 wait complete. 2024-09-19T04:03:43.677 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:43.677 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.677 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.678 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:43.678 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:43.678 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.678 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:43.678 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.678 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:43 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:43.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:43.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:43.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:43.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:43 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.704 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T04:03:43.707 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T04:03:43.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:03:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:43.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:43.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:43.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:43 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:43.977 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:03:44.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 -- 172.21.15.19:0/2284894811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4102660 msgr2=0x7f29b4104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 --2- 172.21.15.19:0/2284894811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4102660 0x7f29b4104a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009900 tx=0x7f29b002a410 comp rx=0 tx=0).stop 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 -- 172.21.15.19:0/2284894811 shutdown_connections 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 --2- 172.21.15.19:0/2284894811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29b41078c0 0x7f29b4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 --2- 172.21.15.19:0/2284894811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b4104f90 0x7f29b4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 --2- 172.21.15.19:0/2284894811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4102660 0x7f29b4104a50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 -- 172.21.15.19:0/2284894811 >> 172.21.15.19:0/2284894811 conn(0x7f29b40fc280 msgr2=0x7f29b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 -- 172.21.15.19:0/2284894811 shutdown_connections 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.611+0000 7f29bc41a640 1 -- 172.21.15.19:0/2284894811 wait complete. 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29bc41a640 1 Processor -- start 2024-09-19T04:03:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29bc41a640 1 -- start start 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29bc41a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b4102660 0x7f29b4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29bc41a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4104f90 0x7f29b4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29bc41a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29b41078c0 0x7f29b419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29bc41a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410d890 con 0x7f29b4104f90 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29bc41a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410d710 con 0x7f29b41078c0 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29bc41a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410da10 con 0x7f29b4102660 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29b99b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4104f90 0x7f29b4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29ba1b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b4102660 0x7f29b4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29b99b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4104f90 0x7f29b4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46506/0 (socket says 172.21.15.19:46506) 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29b99b5640 1 -- 172.21.15.19:0/3249861562 learned_addr learned my addr 172.21.15.19:0/3249861562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:03:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.612+0000 7f29ba9b7640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29b41078c0 0x7f29b419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29b99b5640 1 -- 172.21.15.19:0/3249861562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b4102660 msgr2=0x7f29b4197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29b99b5640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b4102660 0x7f29b4197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29b99b5640 1 -- 172.21.15.19:0/3249861562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29b41078c0 msgr2=0x7f29b419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29b99b5640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29b41078c0 0x7f29b419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29b99b5640 1 -- 172.21.15.19:0/3249861562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b419faf0 con 0x7f29b4104f90 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29b99b5640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4104f90 0x7f29b4198400 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f29a400e9f0 tx=0x7f29a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29ab7fe640 1 -- 172.21.15.19:0/3249861562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a400ccc0 con 0x7f29b4104f90 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b419fde0 con 0x7f29b4104f90 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29ab7fe640 1 -- 172.21.15.19:0/3249861562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a4005500 con 0x7f29b4104f90 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29ab7fe640 1 -- 172.21.15.19:0/3249861562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4010690 con 0x7f29b4104f90 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.613+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b41a0240 con 0x7f29b4104f90 2024-09-19T04:03:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.614+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b40fdd70 con 0x7f29b4104f90 2024-09-19T04:03:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.615+0000 7f29ab7fe640 1 -- 172.21.15.19:0/3249861562 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f29a40046c0 con 0x7f29b4104f90 2024-09-19T04:03:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.616+0000 7f29ab7fe640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29a00a3270 0x7f29a00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.616+0000 7f29ba1b6640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29a00a3270 0x7f29a00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.616+0000 7f29ab7fe640 1 -- 172.21.15.19:0/3249861562 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6809+0+0 (secure 0 0 0) 0x7f29a40cf630 con 0x7f29b4104f90 2024-09-19T04:03:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.617+0000 7f29ba1b6640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29a00a3270 0x7f29a00a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f29b00314e0 tx=0x7f29b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.621+0000 7f29ab7fe640 1 -- 172.21.15.19:0/3249861562 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29a409c930 con 0x7f29b4104f90 2024-09-19T04:03:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.796+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f29b402d970 con 0x7f29b4104f90 2024-09-19T04:03:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.797+0000 7f29ab7fe640 1 -- 172.21.15.19:0/3249861562 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f29a409c230 con 0x7f29b4104f90 2024-09-19T04:03:44.798 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:03:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29a00a3270 msgr2=0x7f29a00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29a00a3270 0x7f29a00a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f29b00314e0 tx=0x7f29b0000bc0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4104f90 msgr2=0x7f29b4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4104f90 0x7f29b4198400 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f29a400e9f0 tx=0x7f29a400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 shutdown_connections 2024-09-19T04:03:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29a00a3270 0x7f29a00a5730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29b41078c0 0x7f29b419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4104f90 0x7f29b4198400 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 --2- 172.21.15.19:0/3249861562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b4102660 0x7f29b4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 >> 172.21.15.19:0/3249861562 conn(0x7f29b40fc280 msgr2=0x7f29b4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 shutdown_connections 2024-09-19T04:03:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:44.799+0000 7f29bc41a640 1 -- 172.21.15.19:0/3249861562 wait complete. 2024-09-19T04:03:45.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:45 smithi019 ceph-mon[30938]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:45.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:45 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3249861562' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:45.617 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726718602,"num_in_osds":12,"osd_in_since":1726718604,"num_remapped_pgs":0} 2024-09-19T04:03:45.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:45 smithi196 ceph-mon[35731]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:45.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:45 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3249861562' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:45.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:45 smithi111 ceph-mon[35365]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:45.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:45 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3249861562' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:45.958 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:03:45 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:03:45.682+0000 7f59a96a9640 -1 osd.11 0 log_to_monitors true 2024-09-19T04:03:46.618 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:03:46.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:46 smithi019 ceph-mon[30938]: from='osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:03:46.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:46 smithi019 ceph-mon[30938]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:03:46.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:46 smithi196 ceph-mon[35731]: from='osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:03:46.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:46 smithi196 ceph-mon[35731]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:03:46.888 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:03:46.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:46 smithi111 ceph-mon[35365]: from='osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:03:46.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:46 smithi111 ceph-mon[35365]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T04:03:47.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 -- 172.21.15.19:0/1257869380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 msgr2=0x7fea6010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:47.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 --2- 172.21.15.19:0/1257869380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 0x7fea6010a050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fea58009ca0 tx=0x7fea5802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 -- 172.21.15.19:0/1257869380 shutdown_connections 2024-09-19T04:03:47.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 --2- 172.21.15.19:0/1257869380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 0x7fea6010a050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 --2- 172.21.15.19:0/1257869380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea60102e00 0x7fea60103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 --2- 172.21.15.19:0/1257869380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea60101c00 0x7fea60102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 -- 172.21.15.19:0/1257869380 >> 172.21.15.19:0/1257869380 conn(0x7fea600fd390 msgr2=0x7fea600ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 -- 172.21.15.19:0/1257869380 shutdown_connections 2024-09-19T04:03:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 -- 172.21.15.19:0/1257869380 wait complete. 2024-09-19T04:03:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 Processor -- start 2024-09-19T04:03:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.357+0000 7fea67f4a640 1 -- start start 2024-09-19T04:03:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea67f4a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea60101c00 0x7fea60197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea67f4a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea60102e00 0x7fea601983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea67f4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 0x7fea6019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea67f4a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea6010d880 con 0x7fea601037c0 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea67f4a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea6010d700 con 0x7fea60102e00 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea67f4a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea6010da00 con 0x7fea60101c00 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 0x7fea6019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 0x7fea6019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46524/0 (socket says 172.21.15.19:46524) 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 -- 172.21.15.19:0/3528221244 learned_addr learned my addr 172.21.15.19:0/3528221244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea654e5640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea60102e00 0x7fea601983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea65ce6640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea60101c00 0x7fea60197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 -- 172.21.15.19:0/3528221244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea60101c00 msgr2=0x7fea60197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea60101c00 0x7fea60197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 -- 172.21.15.19:0/3528221244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea60102e00 msgr2=0x7fea601983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea60102e00 0x7fea601983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 -- 172.21.15.19:0/3528221244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6019fad0 con 0x7fea601037c0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea65ce6640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea60101c00 0x7fea60197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea664e7640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 0x7fea6019f460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea58005e30 tx=0x7fea58031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.358+0000 7fea52ffd640 1 -- 172.21.15.19:0/3528221244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58003c30 con 0x7fea601037c0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.359+0000 7fea52ffd640 1 -- 172.21.15.19:0/3528221244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea58003dd0 con 0x7fea601037c0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.359+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6019fd60 con 0x7fea601037c0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.359+0000 7fea52ffd640 1 -- 172.21.15.19:0/3528221244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5802fde0 con 0x7fea601037c0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.359+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea601a0240 con 0x7fea601037c0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.360+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea60108f80 con 0x7fea601037c0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.361+0000 7fea52ffd640 1 -- 172.21.15.19:0/3528221244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fea58004050 con 0x7fea601037c0 2024-09-19T04:03:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.361+0000 7fea52ffd640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fea4c0a3270 0x7fea4c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:47.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.361+0000 7fea65ce6640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fea4c0a3270 0x7fea4c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:47.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.362+0000 7fea52ffd640 1 -- 172.21.15.19:0/3528221244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6831+0+0 (secure 0 0 0) 0x7fea580f0570 con 0x7fea601037c0 2024-09-19T04:03:47.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.362+0000 7fea65ce6640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fea4c0a3270 0x7fea4c0a5730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0058c0 tx=0x7fea5c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:47.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.366+0000 7fea52ffd640 1 -- 172.21.15.19:0/3528221244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fea580bd860 con 0x7fea601037c0 2024-09-19T04:03:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.543+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fea60062e90 con 0x7fea601037c0 2024-09-19T04:03:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.544+0000 7fea52ffd640 1 -- 172.21.15.19:0/3528221244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fea580bd160 con 0x7fea601037c0 2024-09-19T04:03:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: Detected new or changed devices on smithi196 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:47.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:47 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fea4c0a3270 msgr2=0x7fea4c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:47.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fea4c0a3270 0x7fea4c0a5730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0058c0 tx=0x7fea5c005850 comp rx=0 tx=0).stop 2024-09-19T04:03:47.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 msgr2=0x7fea6019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 0x7fea6019f460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea58005e30 tx=0x7fea58031ee0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 shutdown_connections 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fea4c0a3270 0x7fea4c0a5730 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea601037c0 0x7fea6019f460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea60102e00 0x7fea601983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 --2- 172.21.15.19:0/3528221244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea60101c00 0x7fea60197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 >> 172.21.15.19:0/3528221244 conn(0x7fea600fd390 msgr2=0x7fea60106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 shutdown_connections 2024-09-19T04:03:47.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:47.546+0000 7fea67f4a640 1 -- 172.21.15.19:0/3528221244 wait complete. 2024-09-19T04:03:47.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:03:47.442+0000 7f59a562c640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T04:03:47.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:03:47.454+0000 7f59a1454640 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: Detected new or changed devices on smithi196 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-19T04:03:47.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:47.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:47.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:47 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:47.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T04:03:47.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T04:03:47.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: Detected new or changed devices on smithi196 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:47.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:47 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:48.213 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1726718602,"num_in_osds":12,"osd_in_since":1726718604,"num_remapped_pgs":0} 2024-09-19T04:03:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:48 smithi196 ceph-mon[35731]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:03:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:48 smithi196 ceph-mon[35731]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T04:03:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:48 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:48.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:48 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3528221244' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:48.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:48 smithi019 ceph-mon[30938]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:03:48.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:48 smithi019 ceph-mon[30938]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T04:03:48.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:48.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:48 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:48.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:48 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3528221244' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:48 smithi111 ceph-mon[35365]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T04:03:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:48 smithi111 ceph-mon[35365]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T04:03:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:48 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:48 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3528221244' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:49.215 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T04:03:49.488 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:03:49.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:49 smithi019 ceph-mon[30938]: purged_snaps scrub starts 2024-09-19T04:03:49.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:49 smithi019 ceph-mon[30938]: purged_snaps scrub ok 2024-09-19T04:03:49.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:49 smithi019 ceph-mon[30938]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:49.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:49.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:49 smithi019 ceph-mon[30938]: osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] boot 2024-09-19T04:03:49.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:49 smithi019 ceph-mon[30938]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:03:49.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:49 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:49 smithi196 ceph-mon[35731]: purged_snaps scrub starts 2024-09-19T04:03:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:49 smithi196 ceph-mon[35731]: purged_snaps scrub ok 2024-09-19T04:03:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:49 smithi196 ceph-mon[35731]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:49.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:49 smithi196 ceph-mon[35731]: osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] boot 2024-09-19T04:03:49.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:49 smithi196 ceph-mon[35731]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:03:49.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:49 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:49.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:49 smithi111 ceph-mon[35365]: purged_snaps scrub starts 2024-09-19T04:03:49.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:49 smithi111 ceph-mon[35365]: purged_snaps scrub ok 2024-09-19T04:03:49.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:49 smithi111 ceph-mon[35365]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T04:03:49.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:49.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:49 smithi111 ceph-mon[35365]: osd.11 [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] boot 2024-09-19T04:03:49.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:49 smithi111 ceph-mon[35365]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T04:03:49.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:49 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T04:03:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 -- 172.21.15.19:0/1743322808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 msgr2=0x7f6258108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 --2- 172.21.15.19:0/1743322808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 0x7f6258108f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6250009ca0 tx=0x7f625002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 -- 172.21.15.19:0/1743322808 shutdown_connections 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 --2- 172.21.15.19:0/1743322808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 0x7f6258108f20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 --2- 172.21.15.19:0/1743322808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6258101cd0 0x7f6258102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 --2- 172.21.15.19:0/1743322808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6258100ad0 0x7f6258100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 -- 172.21.15.19:0/1743322808 >> 172.21.15.19:0/1743322808 conn(0x7f62580fc280 msgr2=0x7f62580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 -- 172.21.15.19:0/1743322808 shutdown_connections 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 -- 172.21.15.19:0/1743322808 wait complete. 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.974+0000 7f625ddbe640 1 Processor -- start 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f625ddbe640 1 -- start start 2024-09-19T04:03:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f625ddbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6258100ad0 0x7f625819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f625ddbe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6258101cd0 0x7f625819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f625ddbe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 0x7f62581a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f625ddbe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f625810c780 con 0x7f6258102690 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f625ddbe640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f625810c600 con 0x7f6258101cd0 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f625ddbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f625810c900 con 0x7f6258100ad0 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 0x7f62581a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 0x7f62581a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46548/0 (socket says 172.21.15.19:46548) 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 -- 172.21.15.19:0/3210827614 learned_addr learned my addr 172.21.15.19:0/3210827614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f62577fe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6258100ad0 0x7f625819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6256ffd640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6258101cd0 0x7f625819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 -- 172.21.15.19:0/3210827614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6258100ad0 msgr2=0x7f625819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6258100ad0 0x7f625819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 -- 172.21.15.19:0/3210827614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6258101cd0 msgr2=0x7f625819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6258101cd0 0x7f625819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 -- 172.21.15.19:0/3210827614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62581a1c40 con 0x7f6258102690 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6257fff640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 0x7f62581a15d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f62500319f0 tx=0x7f625002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.975+0000 7f6254ff9640 1 -- 172.21.15.19:0/3210827614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250003c70 con 0x7f6258102690 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.976+0000 7f6254ff9640 1 -- 172.21.15.19:0/3210827614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6250003e10 con 0x7f6258102690 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.976+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62581a1ed0 con 0x7f6258102690 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.976+0000 7f6254ff9640 1 -- 172.21.15.19:0/3210827614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625002fd30 con 0x7f6258102690 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.976+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62581a2380 con 0x7f6258102690 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.976+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62581a26a0 con 0x7f6258102690 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.978+0000 7f6254ff9640 1 -- 172.21.15.19:0/3210827614 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6250004b40 con 0x7f6258102690 2024-09-19T04:03:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.978+0000 7f6254ff9640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62400a3270 0x7f62400a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.979+0000 7f6254ff9640 1 -- 172.21.15.19:0/3210827614 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f62500f0ac0 con 0x7f6258102690 2024-09-19T04:03:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.979+0000 7f62577fe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62400a3270 0x7f62400a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.979+0000 7f62577fe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62400a3270 0x7f62400a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6258101b30 tx=0x7f6248005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:49.982+0000 7f6254ff9640 1 -- 172.21.15.19:0/3210827614 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62500bdc80 con 0x7f6258102690 2024-09-19T04:03:50.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.155+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f62581a2990 con 0x7f6258102690 2024-09-19T04:03:50.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.156+0000 7f6254ff9640 1 -- 172.21.15.19:0/3210827614 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f62500bd580 con 0x7f6258102690 2024-09-19T04:03:50.157 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:03:50.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62400a3270 msgr2=0x7f62400a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62400a3270 0x7f62400a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6258101b30 tx=0x7f6248005f90 comp rx=0 tx=0).stop 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 msgr2=0x7f62581a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 0x7f62581a15d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f62500319f0 tx=0x7f625002f450 comp rx=0 tx=0).stop 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 shutdown_connections 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62400a3270 0x7f62400a5730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6258102690 0x7f62581a15d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6258101cd0 0x7f625819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 --2- 172.21.15.19:0/3210827614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6258100ad0 0x7f625819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.158+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 >> 172.21.15.19:0/3210827614 conn(0x7f62580fc280 msgr2=0x7f62580fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.159+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 shutdown_connections 2024-09-19T04:03:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:50.159+0000 7f625ddbe640 1 -- 172.21.15.19:0/3210827614 wait complete. 2024-09-19T04:03:50.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:50 smithi196 ceph-mon[35731]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:03:50.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:50 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3210827614' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:50.806 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:50 smithi111 ceph-mon[35365]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:03:50.806 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:50 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3210827614' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:50.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:50 smithi019 ceph-mon[30938]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T04:03:50.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:50 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3210827614' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T04:03:50.988 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1726718628,"num_in_osds":12,"osd_in_since":1726718604,"num_remapped_pgs":0} 2024-09-19T04:03:50.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:03:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:03:51.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:51 smithi019 ceph-mon[30938]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:51.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:51 smithi196 ceph-mon[35731]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 -- 172.21.15.19:0/913935766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c101400 msgr2=0x7f7d4c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 --2- 172.21.15.19:0/913935766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c101400 0x7f7d4c10d930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009cd0 tx=0x7f7d4402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 -- 172.21.15.19:0/913935766 shutdown_connections 2024-09-19T04:03:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 --2- 172.21.15.19:0/913935766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c101400 0x7f7d4c10d930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 --2- 172.21.15.19:0/913935766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c100a60 0x7f7d4c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 --2- 172.21.15.19:0/913935766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d4c106a60 0x7f7d4c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 -- 172.21.15.19:0/913935766 >> 172.21.15.19:0/913935766 conn(0x7f7d4c0fc780 msgr2=0x7f7d4c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 -- 172.21.15.19:0/913935766 shutdown_connections 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.733+0000 7f7d5283e640 1 -- 172.21.15.19:0/913935766 wait complete. 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d5283e640 1 Processor -- start 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d5283e640 1 -- start start 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d5283e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d4c100a60 0x7f7d4c102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d5283e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c101400 0x7f7d4c102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d5283e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c106a60 0x7f7d4c1033c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d5283e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c110eb0 con 0x7f7d4c106a60 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d5283e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c110d30 con 0x7f7d4c101400 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d5283e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c111030 con 0x7f7d4c100a60 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c106a60 0x7f7d4c1033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c106a60 0x7f7d4c1033c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40996/0 (socket says 172.21.15.19:40996) 2024-09-19T04:03:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 -- 172.21.15.19:0/811622704 learned_addr learned my addr 172.21.15.19:0/811622704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d4bfff640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d4c100a60 0x7f7d4c102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d43fff640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c101400 0x7f7d4c102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 -- 172.21.15.19:0/811622704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d4c100a60 msgr2=0x7f7d4c102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d4c100a60 0x7f7d4c102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 -- 172.21.15.19:0/811622704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c101400 msgr2=0x7f7d4c102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c101400 0x7f7d4c102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 -- 172.21.15.19:0/811622704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4c1a6820 con 0x7f7d4c106a60 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d43fff640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c101400 0x7f7d4c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.734+0000 7f7d50ddb640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c106a60 0x7f7d4c1033c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009ca0 tx=0x7f7d44003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.735+0000 7f7d49ffb640 1 -- 172.21.15.19:0/811622704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d44004240 con 0x7f7d4c106a60 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.735+0000 7f7d49ffb640 1 -- 172.21.15.19:0/811622704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d440305a0 con 0x7f7d4c106a60 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.735+0000 7f7d49ffb640 1 -- 172.21.15.19:0/811622704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4403a660 con 0x7f7d4c106a60 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.735+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4c1a6ab0 con 0x7f7d4c106a60 2024-09-19T04:03:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.735+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4c1a7040 con 0x7f7d4c106a60 2024-09-19T04:03:51.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.736+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4c100ec0 con 0x7f7d4c106a60 2024-09-19T04:03:51.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.737+0000 7f7d49ffb640 1 -- 172.21.15.19:0/811622704 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7d44030750 con 0x7f7d4c106a60 2024-09-19T04:03:51.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.737+0000 7f7d49ffb640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7d380a32c0 0x7f7d380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:51.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.738+0000 7f7d4bfff640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7d380a32c0 0x7f7d380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:51.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.738+0000 7f7d49ffb640 1 -- 172.21.15.19:0/811622704 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f7d4402e030 con 0x7f7d4c106a60 2024-09-19T04:03:51.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.738+0000 7f7d4bfff640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7d380a32c0 0x7f7d380a5780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00ab60 tx=0x7f7d3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:51.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.741+0000 7f7d49ffb640 1 -- 172.21.15.19:0/811622704 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d440bd7e0 con 0x7f7d4c106a60 2024-09-19T04:03:51.822 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:51 smithi111 ceph-mon[35365]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.918+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d4c1a7270 con 0x7f7d4c106a60 2024-09-19T04:03:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.919+0000 7f7d49ffb640 1 -- 172.21.15.19:0/811622704 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+18444 (secure 0 0 0) 0x7f7d440bd0e0 con 0x7f7d4c106a60 2024-09-19T04:03:51.921 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:03:51.921 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":78,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:03:49.445248+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:03:51.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7d380a32c0 msgr2=0x7f7d380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7d380a32c0 0x7f7d380a5780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00ab60 tx=0x7f7d3c00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c106a60 msgr2=0x7f7d4c1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c106a60 0x7f7d4c1033c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009ca0 tx=0x7f7d44003ba0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 shutdown_connections 2024-09-19T04:03:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7d380a32c0 0x7f7d380a5780 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d4c106a60 0x7f7d4c1033c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c101400 0x7f7d4c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 --2- 172.21.15.19:0/811622704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d4c100a60 0x7f7d4c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 >> 172.21.15.19:0/811622704 conn(0x7f7d4c0fc780 msgr2=0x7f7d4c104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 shutdown_connections 2024-09-19T04:03:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:51.922+0000 7f7d5283e640 1 -- 172.21.15.19:0/811622704 wait complete. 2024-09-19T04:03:52.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:52 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/811622704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:03:52.693 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T03:59:58.845133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:03:52.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T04:03:52.882 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:52 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/811622704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:03:52.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:52 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/811622704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:03:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:03:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 -- 172.21.15.19:0/1438515889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 msgr2=0x7efe8410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 --2- 172.21.15.19:0/1438515889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 0x7efe8410bb50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009900 tx=0x7efe6c02a480 comp rx=0 tx=0).stop 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 -- 172.21.15.19:0/1438515889 shutdown_connections 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 --2- 172.21.15.19:0/1438515889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe8410c170 0x7efe8410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 --2- 172.21.15.19:0/1438515889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 0x7efe8410bb50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 --2- 172.21.15.19:0/1438515889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe84102d40 0x7efe84103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 -- 172.21.15.19:0/1438515889 >> 172.21.15.19:0/1438515889 conn(0x7efe840fc7a0 msgr2=0x7efe840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 -- 172.21.15.19:0/1438515889 shutdown_connections 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.444+0000 7efe8992b640 1 -- 172.21.15.19:0/1438515889 wait complete. 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe8992b640 1 Processor -- start 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe8992b640 1 -- start start 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe8992b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102d40 0x7efe84198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe8992b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 0x7efe84198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe8992b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe8410c170 0x7efe8419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe8992b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe84111f70 con 0x7efe84103660 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe8992b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efe84111df0 con 0x7efe84102d40 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe8992b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe841120f0 con 0x7efe8410c170 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 0x7efe84198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 0x7efe84198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41018/0 (socket says 172.21.15.19:41018) 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe82ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102d40 0x7efe84198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe7bfff640 1 -- 172.21.15.19:0/2341844219 learned_addr learned my addr 172.21.15.19:0/2341844219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.445+0000 7efe837fe640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe8410c170 0x7efe8419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe7bfff640 1 -- 172.21.15.19:0/2341844219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe8410c170 msgr2=0x7efe8419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe7bfff640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe8410c170 0x7efe8419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe7bfff640 1 -- 172.21.15.19:0/2341844219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102d40 msgr2=0x7efe84198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe7bfff640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102d40 0x7efe84198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe7bfff640 1 -- 172.21.15.19:0/2341844219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8419d3e0 con 0x7efe84103660 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe7bfff640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 0x7efe84198970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efe6c006010 tx=0x7efe6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe88929640 1 -- 172.21.15.19:0/2341844219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c03f030 con 0x7efe84103660 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe88929640 1 -- 172.21.15.19:0/2341844219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c004440 con 0x7efe84103660 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe88929640 1 -- 172.21.15.19:0/2341844219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c035070 con 0x7efe84103660 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8419d670 con 0x7efe84103660 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.446+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8419db00 con 0x7efe84103660 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.447+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe84103120 con 0x7efe84103660 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.448+0000 7efe88929640 1 -- 172.21.15.19:0/2341844219 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efe6c042030 con 0x7efe84103660 2024-09-19T04:03:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.449+0000 7efe88929640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efe700a32c0 0x7efe700a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.449+0000 7efe82ffd640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efe700a32c0 0x7efe700a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.449+0000 7efe88929640 1 -- 172.21.15.19:0/2341844219 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7efe6c0ef7d0 con 0x7efe84103660 2024-09-19T04:03:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.450+0000 7efe82ffd640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efe700a32c0 0x7efe700a5780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe74009bd0 tx=0x7efe74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.453+0000 7efe88929640 1 -- 172.21.15.19:0/2341844219 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe6c0bc2a0 con 0x7efe84103660 2024-09-19T04:03:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.630+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe8402d7c0 con 0x7efe84103660 2024-09-19T04:03:53.632 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:53 smithi019 ceph-mon[30938]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.631+0000 7efe88929640 1 -- 172.21.15.19:0/2341844219 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7efe6c0c05a0 con 0x7efe84103660 2024-09-19T04:03:53.633 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efe700a32c0 msgr2=0x7efe700a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efe700a32c0 0x7efe700a5780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe74009bd0 tx=0x7efe74005f90 comp rx=0 tx=0).stop 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 msgr2=0x7efe84198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 0x7efe84198970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efe6c006010 tx=0x7efe6c004040 comp rx=0 tx=0).stop 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 shutdown_connections 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efe700a32c0 0x7efe700a5780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe8410c170 0x7efe8419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe84103660 0x7efe84198970 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 --2- 172.21.15.19:0/2341844219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102d40 0x7efe84198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 >> 172.21.15.19:0/2341844219 conn(0x7efe840fc7a0 msgr2=0x7efe840fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 shutdown_connections 2024-09-19T04:03:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:53.633+0000 7efe8992b640 1 -- 172.21.15.19:0/2341844219 wait complete. 2024-09-19T04:03:53.837 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:53 smithi196 ceph-mon[35731]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:53.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:53 smithi111 ceph-mon[35365]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:54.437 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.b on smithi111 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.e on smithi111 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.h on smithi111 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.k on smithi111 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-09-19T04:03:54.438 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-09-19T04:03:54.439 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-09-19T04:03:54.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:54 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2341844219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:03:54.688 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:03:54.809 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:54 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2341844219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:03:54.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:54 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2341844219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:03:55.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 -- 172.21.15.196:0/452945876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 msgr2=0x7f9d40103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 --2- 172.21.15.196:0/452945876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 0x7f9d40103290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009900 tx=0x7f9d2c02a410 comp rx=0 tx=0).stop 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 -- 172.21.15.196:0/452945876 shutdown_connections 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 --2- 172.21.15.196:0/452945876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d401037d0 0x7f9d4010a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 --2- 172.21.15.196:0/452945876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 0x7f9d40103290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 --2- 172.21.15.196:0/452945876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d40101c10 0x7f9d40102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 -- 172.21.15.196:0/452945876 >> 172.21.15.196:0/452945876 conn(0x7f9d400fd3c0 msgr2=0x7f9d400ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 -- 172.21.15.196:0/452945876 shutdown_connections 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.162+0000 7f9d47232640 1 -- 172.21.15.196:0/452945876 wait complete. 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d47232640 1 Processor -- start 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d47232640 1 -- start start 2024-09-19T04:03:55.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d47232640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d40101c10 0x7f9d40197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d47232640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 0x7f9d401984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d47232640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d401037d0 0x7f9d4019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d47232640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4010d8f0 con 0x7f9d401037d0 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d47232640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4010d770 con 0x7f9d40101c10 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d47232640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4010da70 con 0x7f9d40102e10 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 0x7f9d401984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 0x7f9d401984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34070/0 (socket says 172.21.15.196:34070) 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d3ffff640 1 -- 172.21.15.196:0/2229645593 learned_addr learned my addr 172.21.15.196:0/2229645593 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d44fce640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d40101c10 0x7f9d40197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.163+0000 7f9d457cf640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d401037d0 0x7f9d4019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3ffff640 1 -- 172.21.15.196:0/2229645593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d40101c10 msgr2=0x7f9d40197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:55.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3ffff640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d40101c10 0x7f9d40197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3ffff640 1 -- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d401037d0 msgr2=0x7f9d4019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3ffff640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d401037d0 0x7f9d4019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3ffff640 1 -- 172.21.15.196:0/2229645593 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4019fbb0 con 0x7f9d40102e10 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d44fce640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d40101c10 0x7f9d40197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3ffff640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 0x7f9d401984c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009230 tx=0x7f9d2c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3dffb640 1 -- 172.21.15.196:0/2229645593 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c03f030 con 0x7f9d40102e10 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3dffb640 1 -- 172.21.15.196:0/2229645593 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2c003ce0 con 0x7f9d40102e10 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4019fe40 con 0x7f9d40102e10 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d3dffb640 1 -- 172.21.15.196:0/2229645593 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c035070 con 0x7f9d40102e10 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.164+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d401a0310 con 0x7f9d40102e10 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.165+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d40109060 con 0x7f9d40102e10 2024-09-19T04:03:55.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.166+0000 7f9d3dffb640 1 -- 172.21.15.196:0/2229645593 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9d2c042030 con 0x7f9d40102e10 2024-09-19T04:03:55.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.167+0000 7f9d3dffb640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9d300a3270 0x7f9d300a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:55.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.167+0000 7f9d3dffb640 1 -- 172.21.15.196:0/2229645593 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f9d2c0eff20 con 0x7f9d40102e10 2024-09-19T04:03:55.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.167+0000 7f9d44fce640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9d300a3270 0x7f9d300a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:55.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.168+0000 7f9d44fce640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9d300a3270 0x7f9d300a5730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d340058c0 tx=0x7f9d34005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:55.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.170+0000 7f9d3dffb640 1 -- 172.21.15.196:0/2229645593 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d2c0bd0e0 con 0x7f9d40102e10 2024-09-19T04:03:55.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.347+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7f9d400627f0 con 0x7f9d300a3270 2024-09-19T04:03:55.356 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-09-19T04:03:55.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.355+0000 7f9d3dffb640 1 -- 172.21.15.196:0/2229645593 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9d400627f0 con 0x7f9d300a3270 2024-09-19T04:03:55.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.357+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9d300a3270 msgr2=0x7f9d300a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:55.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.357+0000 7f9d47232640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9d300a3270 0x7f9d300a5730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d340058c0 tx=0x7f9d34005850 comp rx=0 tx=0).stop 2024-09-19T04:03:55.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.357+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 msgr2=0x7f9d401984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:55.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.357+0000 7f9d47232640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 0x7f9d401984c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009230 tx=0x7f9d2c004360 comp rx=0 tx=0).stop 2024-09-19T04:03:55.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.358+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 shutdown_connections 2024-09-19T04:03:55.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.358+0000 7f9d47232640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9d300a3270 0x7f9d300a5730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.358+0000 7f9d47232640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d401037d0 0x7f9d4019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.358+0000 7f9d47232640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d40102e10 0x7f9d401984c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.358+0000 7f9d47232640 1 --2- 172.21.15.196:0/2229645593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d40101c10 0x7f9d40197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:55.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.358+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 >> 172.21.15.196:0/2229645593 conn(0x7f9d400fd3c0 msgr2=0x7f9d40106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:55.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.358+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 shutdown_connections 2024-09-19T04:03:55.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T04:03:55.358+0000 7f9d47232640 1 -- 172.21.15.196:0/2229645593 wait complete. 2024-09-19T04:03:56.013 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T04:03:56.016 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T04:03:56.059 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T04:03:56.102 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T04:03:56.145 DEBUG:teuthology.orchestra.run.smithi111:mds.b> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T04:03:56.147 DEBUG:teuthology.orchestra.run.smithi111:mds.e> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T04:03:56.190 DEBUG:teuthology.orchestra.run.smithi111:mds.h> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T04:03:56.233 DEBUG:teuthology.orchestra.run.smithi111:mds.k> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T04:03:56.276 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T04:03:56.279 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T04:03:56.322 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T04:03:56.365 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T04:03:56.408 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T04:03:56.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:03:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:03:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-19T04:03:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:03:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:03:56.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:56 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:56.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:56.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:03:56.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-19T04:03:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:03:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:03:56.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:56 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:56.663 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:03:56.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:56.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:03:56.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-19T04:03:56.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:56.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:03:56.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:56.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:03:56.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:56.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:03:56.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:03:56.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:56 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:57.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 -- 172.21.15.19:0/1919286497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce01036e0 msgr2=0x7f5ce0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 --2- 172.21.15.19:0/1919286497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce01036e0 0x7f5ce0107b90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009930 tx=0x7f5cc802a420 comp rx=0 tx=0).stop 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 -- 172.21.15.19:0/1919286497 shutdown_connections 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 --2- 172.21.15.19:0/1919286497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce01082a0 0x7f5ce010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 --2- 172.21.15.19:0/1919286497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce01036e0 0x7f5ce0107b90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 --2- 172.21.15.19:0/1919286497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ce0102d30 0x7f5ce0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 -- 172.21.15.19:0/1919286497 >> 172.21.15.19:0/1919286497 conn(0x7f5ce00fc7d0 msgr2=0x7f5ce00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 -- 172.21.15.19:0/1919286497 shutdown_connections 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 -- 172.21.15.19:0/1919286497 wait complete. 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 Processor -- start 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.143+0000 7f5ce7227640 1 -- start start 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce7227640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce0102d30 0x7f5ce0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce7227640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ce01036e0 0x7f5ce0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce7227640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce01082a0 0x7f5ce0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce7227640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01120a0 con 0x7f5ce0102d30 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce7227640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0111f20 con 0x7f5ce01082a0 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce7227640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0112220 con 0x7f5ce01036e0 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce0102d30 0x7f5ce0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce0102d30 0x7f5ce0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41042/0 (socket says 172.21.15.19:41042) 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 -- 172.21.15.19:0/798406858 learned_addr learned my addr 172.21.15.19:0/798406858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce57c4640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce01082a0 0x7f5ce0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5cdffff640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ce01036e0 0x7f5ce0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 -- 172.21.15.19:0/798406858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ce01036e0 msgr2=0x7f5ce0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ce01036e0 0x7f5ce0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 -- 172.21.15.19:0/798406858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce01082a0 msgr2=0x7f5ce0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce01082a0 0x7f5ce0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 -- 172.21.15.19:0/798406858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce01a6aa0 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5ce4fc3640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce0102d30 0x7f5ce0100990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e9f0 tx=0x7f5cd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5cddffb640 1 -- 172.21.15.19:0/798406858 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd000cd80 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.144+0000 7f5cddffb640 1 -- 172.21.15.19:0/798406858 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0005500 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.145+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce01a6d90 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.145+0000 7f5cddffb640 1 -- 172.21.15.19:0/798406858 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0010690 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.145+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01a71f0 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.146+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce0103110 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.147+0000 7f5cddffb640 1 -- 172.21.15.19:0/798406858 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5cd00046c0 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.147+0000 7f5cddffb640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ccc0a3240 0x7f5ccc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.147+0000 7f5cddffb640 1 -- 172.21.15.19:0/798406858 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f5cd00cf890 con 0x7f5ce0102d30 2024-09-19T04:03:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.147+0000 7f5cdffff640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ccc0a3240 0x7f5ccc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:03:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.148+0000 7f5cdffff640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ccc0a3240 0x7f5ccc0a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005940 tx=0x7f5cc80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:03:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.152+0000 7f5cddffb640 1 -- 172.21.15.19:0/798406858 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5cd009ca50 con 0x7f5ce0102d30 2024-09-19T04:03:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.367+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ce010a2b0 con 0x7f5ce0102d30 2024-09-19T04:03:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.367+0000 7f5cddffb640 1 -- 172.21.15.19:0/798406858 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f5cd009c350 con 0x7f5ce0102d30 2024-09-19T04:03:57.369 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:03:57.369 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"btime":"2024-09-19T03:56:42:596148+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T04:03:57.369 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 1 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ccc0a3240 msgr2=0x7f5ccc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ccc0a3240 0x7f5ccc0a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005940 tx=0x7f5cc80058d0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce0102d30 msgr2=0x7f5ce0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce0102d30 0x7f5ce0100990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e9f0 tx=0x7f5cd000edf0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 shutdown_connections 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ccc0a3240 0x7f5ccc0a5700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce01082a0 0x7f5ce0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ce01036e0 0x7f5ce0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 --2- 172.21.15.19:0/798406858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ce0102d30 0x7f5ce0100990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 >> 172.21.15.19:0/798406858 conn(0x7f5ce00fc7d0 msgr2=0x7f5ce0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:03:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 shutdown_connections 2024-09-19T04:03:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:03:57.369+0000 7f5ce7227640 1 -- 172.21.15.19:0/798406858 wait complete. 2024-09-19T04:03:57.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:57 smithi196 ceph-mon[35731]: Deploying daemon mds.l on smithi196 2024-09-19T04:03:57.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:57 smithi111 ceph-mon[35365]: Deploying daemon mds.l on smithi196 2024-09-19T04:03:57.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:57 smithi019 ceph-mon[30938]: Deploying daemon mds.l on smithi196 2024-09-19T04:03:57.958 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:57 smithi196 systemd[1]: Starting Ceph mds.l for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:03:58.248 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:57 smithi196 podman[64866]: 2024-09-19 04:03:57.860742668 +0000 UTC m=+0.015665094 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:03:58.248 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:57 smithi196 podman[64866]: 2024-09-19 04:03:57.977524926 +0000 UTC m=+0.132447360 container create ec465ec7efa530e2c8b8f765cc3225bfe49bbc777077b77960901776759d420a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:03:58.248 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:58 smithi196 podman[64866]: 2024-09-19 04:03:58.246977181 +0000 UTC m=+0.401899608 container init ec465ec7efa530e2c8b8f765cc3225bfe49bbc777077b77960901776759d420a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:03:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:58 smithi019 ceph-mon[30938]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:58 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/798406858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:03:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:58 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6832/2394579385,v1:172.21.15.196:6833/2394579385] up:boot 2024-09-19T04:03:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:58 smithi019 ceph-mon[30938]: fsmap 1 up:standby 2024-09-19T04:03:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:58 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:03:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:58 smithi111 ceph-mon[35365]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:58 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/798406858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:03:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:58 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6832/2394579385,v1:172.21.15.196:6833/2394579385] up:boot 2024-09-19T04:03:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:58 smithi111 ceph-mon[35365]: fsmap 1 up:standby 2024-09-19T04:03:58.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:58 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:03:58.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:58 smithi196 ceph-mon[35731]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:58.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:58 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/798406858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:03:58.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:58 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6832/2394579385,v1:172.21.15.196:6833/2394579385] up:boot 2024-09-19T04:03:58.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:58 smithi196 ceph-mon[35731]: fsmap 1 up:standby 2024-09-19T04:03:58.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:58 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:03:58.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:58 smithi196 podman[64866]: 2024-09-19 04:03:58.25001068 +0000 UTC m=+0.404933109 container start ec465ec7efa530e2c8b8f765cc3225bfe49bbc777077b77960901776759d420a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:03:58.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:58 smithi196 ceph-mds[64886]: -- 172.21.15.196:0/3690035592 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560e5ac0bc20 con 0x560e5abe9400 2024-09-19T04:03:58.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l[64882]: starting mds.l at 2024-09-19T04:03:58.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:58 smithi196 bash[64866]: ec465ec7efa530e2c8b8f765cc3225bfe49bbc777077b77960901776759d420a 2024-09-19T04:03:58.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:03:58 smithi196 systemd[1]: Started Ceph mds.l for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:03:59.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:59 smithi111 ceph-mon[35365]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:59.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:03:59.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:03:59.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:59 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:59.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:03:59 smithi111 ceph-mon[35365]: Deploying daemon mds.k on smithi111 2024-09-19T04:03:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:59 smithi019 ceph-mon[30938]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:03:59.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:03:59.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:59 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:59.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:03:59 smithi019 ceph-mon[30938]: Deploying daemon mds.k on smithi111 2024-09-19T04:03:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:59 smithi196 ceph-mon[35731]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:03:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:03:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:03:59.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:03:59.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:59 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:03:59.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:03:59 smithi196 ceph-mon[35731]: Deploying daemon mds.k on smithi111 2024-09-19T04:04:00.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:00.655 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:00 smithi111 systemd[1]: Starting Ceph mds.k for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 -- 172.21.15.19:0/1256293230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 msgr2=0x7fbfd01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 --2- 172.21.15.19:0/1256293230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 0x7fbfd01029b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009900 tx=0x7fbfc802a480 comp rx=0 tx=0).stop 2024-09-19T04:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 -- 172.21.15.19:0/1256293230 shutdown_connections 2024-09-19T04:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 --2- 172.21.15.19:0/1256293230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd0102ef0 0x7fbfd010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 --2- 172.21.15.19:0/1256293230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 0x7fbfd01029b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 --2- 172.21.15.19:0/1256293230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfd0100e20 0x7fbfd0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 -- 172.21.15.19:0/1256293230 >> 172.21.15.19:0/1256293230 conn(0x7fbfd00fc7d0 msgr2=0x7fbfd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 -- 172.21.15.19:0/1256293230 shutdown_connections 2024-09-19T04:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.919+0000 7fbfd8800640 1 -- 172.21.15.19:0/1256293230 wait complete. 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd8800640 1 Processor -- start 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd8800640 1 -- start start 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd8800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfd0100e20 0x7fbfd01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 0x7fbfd0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd8800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd0102ef0 0x7fbfd019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd8800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0110f80 con 0x7fbfd0102570 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd8800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0110e00 con 0x7fbfd0102ef0 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd8800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0111100 con 0x7fbfd0100e20 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 0x7fbfd0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:00.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 0x7fbfd0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41062/0 (socket says 172.21.15.19:41062) 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd5d9b640 1 -- 172.21.15.19:0/708900441 learned_addr learned my addr 172.21.15.19:0/708900441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd6d9d640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd0102ef0 0x7fbfd019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd659c640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfd0100e20 0x7fbfd01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd5d9b640 1 -- 172.21.15.19:0/708900441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfd0100e20 msgr2=0x7fbfd01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd5d9b640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfd0100e20 0x7fbfd01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd5d9b640 1 -- 172.21.15.19:0/708900441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd0102ef0 msgr2=0x7fbfd019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd5d9b640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd0102ef0 0x7fbfd019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.920+0000 7fbfd5d9b640 1 -- 172.21.15.19:0/708900441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd019d490 con 0x7fbfd0102570 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.921+0000 7fbfd5d9b640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 0x7fbfd0198a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8006010 tx=0x7fbfc8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.921+0000 7fbfc37fe640 1 -- 172.21.15.19:0/708900441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc803f070 con 0x7fbfd0102570 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.921+0000 7fbfc37fe640 1 -- 172.21.15.19:0/708900441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc8031c30 con 0x7fbfd0102570 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.921+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd019d720 con 0x7fbfd0102570 2024-09-19T04:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.921+0000 7fbfc37fe640 1 -- 172.21.15.19:0/708900441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8035030 con 0x7fbfd0102570 2024-09-19T04:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.921+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd01a5060 con 0x7fbfd0102570 2024-09-19T04:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.922+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd0101200 con 0x7fbfd0102570 2024-09-19T04:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.923+0000 7fbfc37fe640 1 -- 172.21.15.19:0/708900441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbfc8042070 con 0x7fbfd0102570 2024-09-19T04:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.924+0000 7fbfc37fe640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbfbc0a32c0 0x7fbfbc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.924+0000 7fbfc37fe640 1 -- 172.21.15.19:0/708900441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fbfc80efc50 con 0x7fbfd0102570 2024-09-19T04:04:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.924+0000 7fbfd659c640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbfbc0a32c0 0x7fbfbc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.924+0000 7fbfd659c640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbfbc0a32c0 0x7fbfbc0a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4005e10 tx=0x7fbfc4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:00.928+0000 7fbfc37fe640 1 -- 172.21.15.19:0/708900441 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbfc80f5020 con 0x7fbfd0102570 2024-09-19T04:04:01.023 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:00 smithi111 podman[64369]: 2024-09-19 04:04:00.609862092 +0000 UTC m=+0.015707629 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:01.023 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:00 smithi111 podman[64369]: 2024-09-19 04:04:00.738519865 +0000 UTC m=+0.144365409 container create 120cf7baa660cb2da2c1e0354e05d80181056525be39b54e257ef0971e735b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:04:01.024 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:01 smithi111 podman[64369]: 2024-09-19 04:04:01.023048157 +0000 UTC m=+0.428893695 container init 120cf7baa660cb2da2c1e0354e05d80181056525be39b54e257ef0971e735b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:04:01.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.144+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfd010a0a0 con 0x7fbfd0102570 2024-09-19T04:04:01.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.145+0000 7fbfc37fe640 1 -- 172.21.15.19:0/708900441 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fbfc8031de0 con 0x7fbfd0102570 2024-09-19T04:04:01.147 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:01.147 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"btime":"2024-09-19T04:03:58:278136+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:04:01.147 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-09-19T04:04:01.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbfbc0a32c0 msgr2=0x7fbfbc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:01.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbfbc0a32c0 0x7fbfbc0a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4005e10 tx=0x7fbfc4005da0 comp rx=0 tx=0).stop 2024-09-19T04:04:01.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 msgr2=0x7fbfd0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:01.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 0x7fbfd0198a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8006010 tx=0x7fbfc8003c10 comp rx=0 tx=0).stop 2024-09-19T04:04:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 shutdown_connections 2024-09-19T04:04:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbfbc0a32c0 0x7fbfbc0a5780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd0102ef0 0x7fbfd019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd0102570 0x7fbfd0198a20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 --2- 172.21.15.19:0/708900441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfd0100e20 0x7fbfd01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 >> 172.21.15.19:0/708900441 conn(0x7fbfd00fc7d0 msgr2=0x7fbfd006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 shutdown_connections 2024-09-19T04:04:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:01.147+0000 7fbfd8800640 1 -- 172.21.15.19:0/708900441 wait complete. 2024-09-19T04:04:01.405 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:01 smithi111 podman[64369]: 2024-09-19 04:04:01.026202316 +0000 UTC m=+0.432047854 container start 120cf7baa660cb2da2c1e0354e05d80181056525be39b54e257ef0971e735b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T04:04:01.405 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mds[64388]: -- 172.21.15.111:0/2317839103 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614f41c5860 con 0x5614f41a3400 2024-09-19T04:04:01.405 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k[64384]: starting mds.k at 2024-09-19T04:04:01.405 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:01 smithi111 bash[64369]: 120cf7baa660cb2da2c1e0354e05d80181056525be39b54e257ef0971e735b4c 2024-09-19T04:04:01.405 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:04:01 smithi111 systemd[1]: Started Ceph mds.k for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:01.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:01.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/708900441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:01.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:01.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:01.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:01.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:01 smithi019 ceph-mon[30938]: Deploying daemon mds.j on smithi019 2024-09-19T04:04:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/708900441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:01.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:01.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:01.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:01 smithi196 ceph-mon[35731]: Deploying daemon mds.j on smithi019 2024-09-19T04:04:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/708900441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:01.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:01.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:01.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:01.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:01 smithi111 ceph-mon[35365]: Deploying daemon mds.j on smithi019 2024-09-19T04:04:02.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:02 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6832/3618359356,v1:172.21.15.111:6833/3618359356] up:boot 2024-09-19T04:04:02.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:02 smithi111 ceph-mon[35365]: fsmap 2 up:standby 2024-09-19T04:04:02.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:02 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:04:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:02 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6832/3618359356,v1:172.21.15.111:6833/3618359356] up:boot 2024-09-19T04:04:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:02 smithi019 ceph-mon[30938]: fsmap 2 up:standby 2024-09-19T04:04:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:02 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:04:02.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:02 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6832/3618359356,v1:172.21.15.111:6833/3618359356] up:boot 2024-09-19T04:04:02.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:02 smithi196 ceph-mon[35731]: fsmap 2 up:standby 2024-09-19T04:04:02.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:02 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:04:03.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:03 smithi019 ceph-mon[30938]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:03.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:03 smithi196 ceph-mon[35731]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:03.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:03 smithi111 ceph-mon[35365]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:03.931 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:03 smithi019 systemd[1]: Starting Ceph mds.j for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:03.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:04.239 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:03 smithi019 podman[69964]: 2024-09-19 04:04:03.968729647 +0000 UTC m=+0.096280976 container create 6387b03525af81f0595d369e2d4102f27c0f5da42b3547f74a3bf196a2f06eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T04:04:04.239 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:03 smithi019 podman[69964]: 2024-09-19 04:04:03.888822427 +0000 UTC m=+0.016373766 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:04.239 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:04 smithi019 podman[69964]: 2024-09-19 04:04:04.238284185 +0000 UTC m=+0.365835517 container init 6387b03525af81f0595d369e2d4102f27c0f5da42b3547f74a3bf196a2f06eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T04:04:04.435 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:04.562 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:04 smithi019 podman[69964]: 2024-09-19 04:04:04.241722985 +0000 UTC m=+0.369274329 container start 6387b03525af81f0595d369e2d4102f27c0f5da42b3547f74a3bf196a2f06eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T04:04:04.562 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:04 smithi019 ceph-mds[70012]: -- 172.21.15.19:0/2223618155 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565526595860 con 0x565526572800 2024-09-19T04:04:04.562 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j[69997]: starting mds.j at 2024-09-19T04:04:04.562 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:04 smithi019 bash[69964]: 6387b03525af81f0595d369e2d4102f27c0f5da42b3547f74a3bf196a2f06eba 2024-09-19T04:04:04.562 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:04:04 smithi019 systemd[1]: Started Ceph mds.j for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:04.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 -- 172.21.15.19:0/287568216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88102790 msgr2=0x7fab881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:04.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 --2- 172.21.15.19:0/287568216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88102790 0x7fab881067e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009900 tx=0x7fab7c02a410 comp rx=0 tx=0).stop 2024-09-19T04:04:04.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 -- 172.21.15.19:0/287568216 shutdown_connections 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 --2- 172.21.15.19:0/287568216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab88106f10 0x7fab8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 --2- 172.21.15.19:0/287568216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88102790 0x7fab881067e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 --2- 172.21.15.19:0/287568216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab88101de0 0x7fab881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 -- 172.21.15.19:0/287568216 >> 172.21.15.19:0/287568216 conn(0x7fab880fd8e0 msgr2=0x7fab880ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 -- 172.21.15.19:0/287568216 shutdown_connections 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.966+0000 7fab8d855640 1 -- 172.21.15.19:0/287568216 wait complete. 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab8d855640 1 Processor -- start 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab8d855640 1 -- start start 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab8d855640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88101de0 0x7fab88198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab8d855640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab88102790 0x7fab88198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab8d855640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab88106f10 0x7fab8819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:04.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab8d855640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab88111fc0 con 0x7fab88101de0 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab8d855640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab88111e40 con 0x7fab88102790 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab8d855640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab88112140 con 0x7fab88106f10 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab86ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88101de0 0x7fab88198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab86ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88101de0 0x7fab88198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58738/0 (socket says 172.21.15.19:58738) 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.967+0000 7fab86ffd640 1 -- 172.21.15.19:0/3609623066 learned_addr learned my addr 172.21.15.19:0/3609623066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab867fc640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab88102790 0x7fab88198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab86ffd640 1 -- 172.21.15.19:0/3609623066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab88106f10 msgr2=0x7fab8819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab877fe640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab88106f10 0x7fab8819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab86ffd640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab88106f10 0x7fab8819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab86ffd640 1 -- 172.21.15.19:0/3609623066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab88102790 msgr2=0x7fab88198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab86ffd640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab88102790 0x7fab88198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:04.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab86ffd640 1 -- 172.21.15.19:0/3609623066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8819d3f0 con 0x7fab88101de0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab877fe640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab88106f10 0x7fab8819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab86ffd640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88101de0 0x7fab88198440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab7800d950 tx=0x7fab7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab8c853640 1 -- 172.21.15.19:0/3609623066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78017070 con 0x7fab88101de0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8819d6e0 con 0x7fab88101de0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab8c853640 1 -- 172.21.15.19:0/3609623066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab78009c30 con 0x7fab88101de0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab8c853640 1 -- 172.21.15.19:0/3609623066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800f6b0 con 0x7fab88101de0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.968+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab881a4fc0 con 0x7fab88101de0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.969+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab881021c0 con 0x7fab88101de0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.970+0000 7fab8c853640 1 -- 172.21.15.19:0/3609623066 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fab780109d0 con 0x7fab88101de0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.971+0000 7fab8c853640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fab700a3240 0x7fab700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.971+0000 7fab867fc640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fab700a3240 0x7fab700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.971+0000 7fab8c853640 1 -- 172.21.15.19:0/3609623066 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fab780cf0a0 con 0x7fab88101de0 2024-09-19T04:04:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.972+0000 7fab867fc640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fab700a3240 0x7fab700a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000c00 tx=0x7fab7c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:04.975+0000 7fab8c853640 1 -- 172.21.15.19:0/3609623066 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fab78009de0 con 0x7fab88101de0 2024-09-19T04:04:05.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.193+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab881997a0 con 0x7fab88101de0 2024-09-19T04:04:05.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.193+0000 7fab8c853640 1 -- 172.21.15.19:0/3609623066 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7fab7809c240 con 0x7fab88101de0 2024-09-19T04:04:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:05.195 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"btime":"2024-09-19T04:04:04:422821+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:04:05.195 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-09-19T04:04:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fab700a3240 msgr2=0x7fab700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fab700a3240 0x7fab700a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000c00 tx=0x7fab7c005b50 comp rx=0 tx=0).stop 2024-09-19T04:04:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88101de0 msgr2=0x7fab88198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88101de0 0x7fab88198440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab7800d950 tx=0x7fab7800de10 comp rx=0 tx=0).stop 2024-09-19T04:04:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 shutdown_connections 2024-09-19T04:04:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fab700a3240 0x7fab700a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab88106f10 0x7fab8819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab88102790 0x7fab88198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 --2- 172.21.15.19:0/3609623066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab88101de0 0x7fab88198440 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 >> 172.21.15.19:0/3609623066 conn(0x7fab880fd8e0 msgr2=0x7fab880fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 shutdown_connections 2024-09-19T04:04:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:05.196+0000 7fab8d855640 1 -- 172.21.15.19:0/3609623066 wait complete. 2024-09-19T04:04:05.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:05.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:05.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:05.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:05.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: Deploying daemon mds.d on smithi019 2024-09-19T04:04:05.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.19:6834/243568683,v1:172.21.15.19:6835/243568683] up:boot 2024-09-19T04:04:05.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: fsmap 3 up:standby 2024-09-19T04:04:05.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:04:05.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:05 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3609623066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: Deploying daemon mds.d on smithi019 2024-09-19T04:04:05.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.19:6834/243568683,v1:172.21.15.19:6835/243568683] up:boot 2024-09-19T04:04:05.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: fsmap 3 up:standby 2024-09-19T04:04:05.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:04:05.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:05 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3609623066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: Deploying daemon mds.d on smithi019 2024-09-19T04:04:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.19:6834/243568683,v1:172.21.15.19:6835/243568683] up:boot 2024-09-19T04:04:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: fsmap 3 up:standby 2024-09-19T04:04:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T04:04:05.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:05 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3609623066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:07.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:07 smithi019 ceph-mon[30938]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:07.681 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:07 smithi019 systemd[1]: Starting Ceph mds.d for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:07.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:07 smithi196 ceph-mon[35731]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:07.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:07 smithi111 ceph-mon[35365]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:08.036 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:07 smithi019 podman[70378]: 2024-09-19 04:04:07.678476256 +0000 UTC m=+0.016577017 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:08.036 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:07 smithi019 podman[70378]: 2024-09-19 04:04:07.790860756 +0000 UTC m=+0.128961495 container create a4342fddd53d162df38821c1467221bef1f8e88a60f4cd1d4194a494f8e80fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:04:08.036 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:08 smithi019 podman[70378]: 2024-09-19 04:04:08.035225265 +0000 UTC m=+0.373326008 container init a4342fddd53d162df38821c1467221bef1f8e88a60f4cd1d4194a494f8e80fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T04:04:08.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:08.373 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:08.398 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:08 smithi019 podman[70378]: 2024-09-19 04:04:08.038360281 +0000 UTC m=+0.376461015 container start a4342fddd53d162df38821c1467221bef1f8e88a60f4cd1d4194a494f8e80fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T04:04:08.399 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:08 smithi019 ceph-mds[70397]: -- 172.21.15.19:0/3867598260 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e78851f860 con 0x55e7884fd400 2024-09-19T04:04:08.399 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:08 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d[70393]: starting mds.d at 2024-09-19T04:04:08.399 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:08 smithi019 bash[70378]: a4342fddd53d162df38821c1467221bef1f8e88a60f4cd1d4194a494f8e80fcb 2024-09-19T04:04:08.399 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:04:08 smithi019 systemd[1]: Started Ceph mds.d for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 -- 172.21.15.19:0/3886591903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe201082a0 msgr2=0x7fbe2010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 --2- 172.21.15.19:0/3886591903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe201082a0 0x7fbe2010ec40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009cd0 tx=0x7fbe1802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:04:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 -- 172.21.15.19:0/3886591903 shutdown_connections 2024-09-19T04:04:08.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 --2- 172.21.15.19:0/3886591903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe201082a0 0x7fbe2010ec40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 --2- 172.21.15.19:0/3886591903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe201036e0 0x7fbe20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 --2- 172.21.15.19:0/3886591903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe20102d30 0x7fbe20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 -- 172.21.15.19:0/3886591903 >> 172.21.15.19:0/3886591903 conn(0x7fbe200fc7d0 msgr2=0x7fbe200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 -- 172.21.15.19:0/3886591903 shutdown_connections 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.956+0000 7fbe25e06640 1 -- 172.21.15.19:0/3886591903 wait complete. 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe25e06640 1 Processor -- start 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe25e06640 1 -- start start 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe25e06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20102d30 0x7fbe201984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe25e06640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe201036e0 0x7fbe20198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe25e06640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe201082a0 0x7fbe2019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe25e06640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201120a0 con 0x7fbe20102d30 2024-09-19T04:04:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe25e06640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20111f20 con 0x7fbe201082a0 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe25e06640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20112220 con 0x7fbe201036e0 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20102d30 0x7fbe201984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20102d30 0x7fbe201984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58778/0 (socket says 172.21.15.19:58778) 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe1f7fe640 1 -- 172.21.15.19:0/2232649546 learned_addr learned my addr 172.21.15.19:0/2232649546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe16ffd640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe201036e0 0x7fbe20198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.957+0000 7fbe1ffff640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe201082a0 0x7fbe2019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1f7fe640 1 -- 172.21.15.19:0/2232649546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe201036e0 msgr2=0x7fbe20198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1f7fe640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe201036e0 0x7fbe20198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1f7fe640 1 -- 172.21.15.19:0/2232649546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe201082a0 msgr2=0x7fbe2019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1f7fe640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe201082a0 0x7fbe2019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1f7fe640 1 -- 172.21.15.19:0/2232649546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2019d4a0 con 0x7fbe20102d30 2024-09-19T04:04:08.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1f7fe640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20102d30 0x7fbe201984f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000d770 tx=0x7fbe1000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1d7fa640 1 -- 172.21.15.19:0/2232649546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10017070 con 0x7fbe20102d30 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1d7fa640 1 -- 172.21.15.19:0/2232649546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe10009820 con 0x7fbe20102d30 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2019d790 con 0x7fbe20102d30 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe1d7fa640 1 -- 172.21.15.19:0/2232649546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10004cd0 con 0x7fbe20102d30 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.958+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe201a5070 con 0x7fbe20102d30 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.959+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe20103110 con 0x7fbe20102d30 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.960+0000 7fbe1d7fa640 1 -- 172.21.15.19:0/2232649546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbe100099d0 con 0x7fbe20102d30 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.961+0000 7fbe1d7fa640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe0c0a32c0 0x7fbe0c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.961+0000 7fbe16ffd640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe0c0a32c0 0x7fbe0c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.961+0000 7fbe1d7fa640 1 -- 172.21.15.19:0/2232649546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fbe100cf2f0 con 0x7fbe20102d30 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.962+0000 7fbe16ffd640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe0c0a32c0 0x7fbe0c0a5780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe200fed60 tx=0x7fbe0800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:08.968+0000 7fbe1d7fa640 1 -- 172.21.15.19:0/2232649546 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbe1009c490 con 0x7fbe20102d30 2024-09-19T04:04:09.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.183+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe2010a2b0 con 0x7fbe20102d30 2024-09-19T04:04:09.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.183+0000 7fbe1d7fa640 1 -- 172.21.15.19:0/2232649546 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fbe1009bd90 con 0x7fbe20102d30 2024-09-19T04:04:09.185 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:09.185 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"btime":"2024-09-19T04:04:08:410672+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:04:09.185 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 5 2024-09-19T04:04:09.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe0c0a32c0 msgr2=0x7fbe0c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:09.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe0c0a32c0 0x7fbe0c0a5780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe200fed60 tx=0x7fbe0800a2d0 comp rx=0 tx=0).stop 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20102d30 msgr2=0x7fbe201984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20102d30 0x7fbe201984f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000d770 tx=0x7fbe1000db70 comp rx=0 tx=0).stop 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 shutdown_connections 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe0c0a32c0 0x7fbe0c0a5780 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe201082a0 0x7fbe2019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe201036e0 0x7fbe20198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 --2- 172.21.15.19:0/2232649546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20102d30 0x7fbe201984f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 >> 172.21.15.19:0/2232649546 conn(0x7fbe200fc7d0 msgr2=0x7fbe20106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 shutdown_connections 2024-09-19T04:04:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:09.186+0000 7fbe25e06640 1 -- 172.21.15.19:0/2232649546 wait complete. 2024-09-19T04:04:09.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:09.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:09.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:09.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: Deploying daemon mds.e on smithi111 2024-09-19T04:04:09.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.19:6836/2162743100,v1:172.21.15.19:6837/2162743100] up:boot 2024-09-19T04:04:09.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: fsmap 4 up:standby 2024-09-19T04:04:09.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:04:09.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:09 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2232649546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:09.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:09.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:09.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:09.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: Deploying daemon mds.e on smithi111 2024-09-19T04:04:09.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.19:6836/2162743100,v1:172.21.15.19:6837/2162743100] up:boot 2024-09-19T04:04:09.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: fsmap 4 up:standby 2024-09-19T04:04:09.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:04:09.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:09 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2232649546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: Deploying daemon mds.e on smithi111 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.19:6836/2162743100,v1:172.21.15.19:6837/2162743100] up:boot 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: fsmap 4 up:standby 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T04:04:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:09 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2232649546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:10 smithi111 ceph-mon[35365]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:10.405 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 systemd[1]: Starting Ceph mds.e for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:10.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:10 smithi019 ceph-mon[30938]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:10.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:10 smithi196 ceph-mon[35731]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:10.796 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 podman[64671]: 2024-09-19 04:04:10.387247896 +0000 UTC m=+0.016285383 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:10.796 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 podman[64671]: 2024-09-19 04:04:10.534130434 +0000 UTC m=+0.163167935 container create afee5bda59ab334380dc7d6e00320c049937f4a12d82a999f88e82ba1a27c31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T04:04:10.796 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 podman[64671]: 2024-09-19 04:04:10.794947174 +0000 UTC m=+0.423984646 container init afee5bda59ab334380dc7d6e00320c049937f4a12d82a999f88e82ba1a27c31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-19T04:04:11.155 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 podman[64671]: 2024-09-19 04:04:10.798603401 +0000 UTC m=+0.427640860 container start afee5bda59ab334380dc7d6e00320c049937f4a12d82a999f88e82ba1a27c31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T04:04:11.155 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 ceph-mds[64690]: -- 172.21.15.111:0/697420752 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56442c595860 con 0x56442c573400 2024-09-19T04:04:11.155 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e[64686]: starting mds.e at 2024-09-19T04:04:11.155 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 bash[64671]: afee5bda59ab334380dc7d6e00320c049937f4a12d82a999f88e82ba1a27c31b 2024-09-19T04:04:11.155 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:04:10 smithi111 systemd[1]: Started Ceph mds.e for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:11.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:12.156 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:12.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:12.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:12.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:12.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:12.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: Deploying daemon mds.f on smithi196 2024-09-19T04:04:12.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6834/3717777626,v1:172.21.15.111:6835/3717777626] up:boot 2024-09-19T04:04:12.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: fsmap 5 up:standby 2024-09-19T04:04:12.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:04:12.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:12.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:12.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:12.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:12.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: Deploying daemon mds.f on smithi196 2024-09-19T04:04:12.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6834/3717777626,v1:172.21.15.111:6835/3717777626] up:boot 2024-09-19T04:04:12.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: fsmap 5 up:standby 2024-09-19T04:04:12.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:04:12.219 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:12.219 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.219 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.219 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:12.219 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:12.219 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:12.220 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:12.220 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: Deploying daemon mds.f on smithi196 2024-09-19T04:04:12.220 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6834/3717777626,v1:172.21.15.111:6835/3717777626] up:boot 2024-09-19T04:04:12.220 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: fsmap 5 up:standby 2024-09-19T04:04:12.220 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:04:12.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 -- 172.21.15.19:0/1956569274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 msgr2=0x7ff5e01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:12.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 --2- 172.21.15.19:0/1956569274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 0x7ff5e01021c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009900 tx=0x7ff5d002a410 comp rx=0 tx=0).stop 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 -- 172.21.15.19:0/1956569274 shutdown_connections 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 --2- 172.21.15.19:0/1956569274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5e0106f10 0x7ff5e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 --2- 172.21.15.19:0/1956569274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e0102790 0x7ff5e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 --2- 172.21.15.19:0/1956569274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 0x7ff5e01021c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 -- 172.21.15.19:0/1956569274 >> 172.21.15.19:0/1956569274 conn(0x7ff5e00fd8c0 msgr2=0x7ff5e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 -- 172.21.15.19:0/1956569274 shutdown_connections 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.645+0000 7ff5e5494640 1 -- 172.21.15.19:0/1956569274 wait complete. 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5e5494640 1 Processor -- start 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5e5494640 1 -- start start 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5e5494640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 0x7ff5e01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5e5494640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e0102790 0x7ff5e0198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5e5494640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5e0106f10 0x7ff5e019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5e5494640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0111fc0 con 0x7ff5e0101de0 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5e5494640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0111e40 con 0x7ff5e0106f10 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5e5494640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0112140 con 0x7ff5e0102790 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 0x7ff5e01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 0x7ff5e01983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45740/0 (socket says 172.21.15.19:45740) 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5deffd640 1 -- 172.21.15.19:0/786008452 learned_addr learned my addr 172.21.15.19:0/786008452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5de7fc640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e0102790 0x7ff5e0198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.646+0000 7ff5df7fe640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5e0106f10 0x7ff5e019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5deffd640 1 -- 172.21.15.19:0/786008452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e0102790 msgr2=0x7ff5e0198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5deffd640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e0102790 0x7ff5e0198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5deffd640 1 -- 172.21.15.19:0/786008452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5e0106f10 msgr2=0x7ff5e019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5deffd640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5e0106f10 0x7ff5e019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5deffd640 1 -- 172.21.15.19:0/786008452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e019d390 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5deffd640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 0x7ff5e01983e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0004770 tx=0x7ff5d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5d7fff640 1 -- 172.21.15.19:0/786008452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d003f070 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5d7fff640 1 -- 172.21.15.19:0/786008452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d0031ca0 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e019d620 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5d7fff640 1 -- 172.21.15.19:0/786008452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0035030 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.647+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e01a4f60 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.648+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e01032c0 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.649+0000 7ff5d7fff640 1 -- 172.21.15.19:0/786008452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff5d0042070 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.650+0000 7ff5d7fff640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5cc0a3240 0x7ff5cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.650+0000 7ff5de7fc640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5cc0a3240 0x7ff5cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.650+0000 7ff5d7fff640 1 -- 172.21.15.19:0/786008452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7ff5d00efbb0 con 0x7ff5e0101de0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.650+0000 7ff5de7fc640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5cc0a3240 0x7ff5cc0a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00fdce0 tx=0x7ff5c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.654+0000 7ff5d7fff640 1 -- 172.21.15.19:0/786008452 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5d00bc720 con 0x7ff5e0101de0 2024-09-19T04:04:12.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.867+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5e010a0f0 con 0x7ff5e0101de0 2024-09-19T04:04:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.867+0000 7ff5d7fff640 1 -- 172.21.15.19:0/786008452 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7ff5d00c0a20 con 0x7ff5e0101de0 2024-09-19T04:04:12.869 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:12.869 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"btime":"2024-09-19T04:04:11:215930+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T04:04:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-09-19T04:04:12.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5cc0a3240 msgr2=0x7ff5cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5cc0a3240 0x7ff5cc0a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00fdce0 tx=0x7ff5c800b390 comp rx=0 tx=0).stop 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 msgr2=0x7ff5e01983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 0x7ff5e01983e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0004770 tx=0x7ff5d0003c10 comp rx=0 tx=0).stop 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 shutdown_connections 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5cc0a3240 0x7ff5cc0a5700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5e0106f10 0x7ff5e019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e0102790 0x7ff5e0198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 --2- 172.21.15.19:0/786008452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e0101de0 0x7ff5e01983e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 >> 172.21.15.19:0/786008452 conn(0x7ff5e00fd8c0 msgr2=0x7ff5e00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 shutdown_connections 2024-09-19T04:04:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:12.871+0000 7ff5e5494640 1 -- 172.21.15.19:0/786008452 wait complete. 2024-09-19T04:04:12.952 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:12 smithi196 systemd[1]: Starting Ceph mds.f for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:13.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:12 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/786008452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:13.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:12 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/786008452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:13.208 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:13 smithi196 podman[65169]: 2024-09-19 04:04:12.966429158 +0000 UTC m=+0.018144997 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:13.208 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:13 smithi196 podman[65169]: 2024-09-19 04:04:13.090366596 +0000 UTC m=+0.142082431 container create e31700601e2aa0a989806ea95969745683473d40fb0d7a4aaeb17c8d05e50e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:04:13.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:12 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/786008452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:13.708 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:13 smithi196 podman[65169]: 2024-09-19 04:04:13.351635288 +0000 UTC m=+0.403351128 container init e31700601e2aa0a989806ea95969745683473d40fb0d7a4aaeb17c8d05e50e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T04:04:13.708 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:13 smithi196 podman[65169]: 2024-09-19 04:04:13.354489385 +0000 UTC m=+0.406205221 container start e31700601e2aa0a989806ea95969745683473d40fb0d7a4aaeb17c8d05e50e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T04:04:13.708 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-mds[65188]: -- 172.21.15.196:0/4147365168 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bec097f860 con 0x55bec095d000 2024-09-19T04:04:13.708 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f[65184]: starting mds.f at 2024-09-19T04:04:13.709 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:13 smithi196 bash[65169]: e31700601e2aa0a989806ea95969745683473d40fb0d7a4aaeb17c8d05e50e85 2024-09-19T04:04:13.709 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:04:13 smithi196 systemd[1]: Started Ceph mds.f for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:14.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-mon[35731]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:14.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:14.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:14.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:13 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:13 smithi111 ceph-mon[35365]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:13 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:13 smithi019 ceph-mon[30938]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:14.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:14.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:14.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:13 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:15.246 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:14 smithi196 ceph-mon[35731]: Deploying daemon mds.c on smithi196 2024-09-19T04:04:15.246 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:14 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] up:boot 2024-09-19T04:04:15.247 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:14 smithi196 ceph-mon[35731]: fsmap 6 up:standby 2024-09-19T04:04:15.247 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:14 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:04:15.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:14 smithi111 ceph-mon[35365]: Deploying daemon mds.c on smithi196 2024-09-19T04:04:15.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:14 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] up:boot 2024-09-19T04:04:15.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:14 smithi111 ceph-mon[35365]: fsmap 6 up:standby 2024-09-19T04:04:15.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:14 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:04:15.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:14 smithi019 ceph-mon[30938]: Deploying daemon mds.c on smithi196 2024-09-19T04:04:15.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:14 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] up:boot 2024-09-19T04:04:15.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:14 smithi019 ceph-mon[30938]: fsmap 6 up:standby 2024-09-19T04:04:15.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:14 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:04:15.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:15.640 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:15 smithi196 systemd[1]: Starting Ceph mds.c for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:15.932 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:15 smithi196 podman[65469]: 2024-09-19 04:04:15.541662928 +0000 UTC m=+0.015696865 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:15.932 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:15 smithi196 podman[65469]: 2024-09-19 04:04:15.66207629 +0000 UTC m=+0.136110259 container create 12a2b6b5c2023b3f2df14e3fc0c89c1a06ff1c4dcc6f65b1c8525a22ce8e94ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T04:04:15.932 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:15 smithi196 podman[65469]: 2024-09-19 04:04:15.931490227 +0000 UTC m=+0.405524150 container init 12a2b6b5c2023b3f2df14e3fc0c89c1a06ff1c4dcc6f65b1c8525a22ce8e94ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T04:04:16.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:15 smithi196 ceph-mon[35731]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:16.208 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:15 smithi196 podman[65469]: 2024-09-19 04:04:15.934533635 +0000 UTC m=+0.408567559 container start 12a2b6b5c2023b3f2df14e3fc0c89c1a06ff1c4dcc6f65b1c8525a22ce8e94ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T04:04:16.208 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:15 smithi196 ceph-mds[65488]: -- 172.21.15.196:0/4286593022 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56046d539860 con 0x56046d517000 2024-09-19T04:04:16.209 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:15 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c[65484]: starting mds.c at 2024-09-19T04:04:16.209 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:16 smithi196 bash[65469]: 12a2b6b5c2023b3f2df14e3fc0c89c1a06ff1c4dcc6f65b1c8525a22ce8e94ce 2024-09-19T04:04:16.209 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:04:16 smithi196 systemd[1]: Started Ceph mds.c for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:16.231 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:15 smithi019 ceph-mon[30938]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.311+0000 7f1cd53ce640 1 -- 172.21.15.19:0/621814100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0106ba0 msgr2=0x7f1cd0106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.311+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/621814100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0106ba0 0x7f1cd0106f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009900 tx=0x7f1cc002a410 comp rx=0 tx=0).stop 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.311+0000 7f1cd53ce640 1 -- 172.21.15.19:0/621814100 shutdown_connections 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.311+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/621814100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cd0100a70 0x7f1cd010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.311+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/621814100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd01000d0 0x7f1cd0100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.311+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/621814100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0106ba0 0x7f1cd0106f80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.311+0000 7f1cd53ce640 1 -- 172.21.15.19:0/621814100 >> 172.21.15.19:0/621814100 conn(0x7f1cd0077210 msgr2=0x7f1cd00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.312+0000 7f1cd53ce640 1 -- 172.21.15.19:0/621814100 shutdown_connections 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.312+0000 7f1cd53ce640 1 -- 172.21.15.19:0/621814100 wait complete. 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.312+0000 7f1cd53ce640 1 Processor -- start 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.312+0000 7f1cd53ce640 1 -- start start 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cd53ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cd01000d0 0x7f1cd0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cd53ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0100a70 0x7f1cd0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cd53ce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd0106ba0 0x7f1cd019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cd53ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0112090 con 0x7f1cd0100a70 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cd53ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0111f10 con 0x7f1cd01000d0 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cd53ce640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0112210 con 0x7f1cd0106ba0 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0100a70 0x7f1cd0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0100a70 0x7f1cd0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45748/0 (socket says 172.21.15.19:45748) 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 -- 172.21.15.19:0/4108698780 learned_addr learned my addr 172.21.15.19:0/4108698780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cceffd640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cd01000d0 0x7f1cd0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1ccf7fe640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd0106ba0 0x7f1cd019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 -- 172.21.15.19:0/4108698780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd0106ba0 msgr2=0x7f1cd019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd0106ba0 0x7f1cd019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 -- 172.21.15.19:0/4108698780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cd01000d0 msgr2=0x7f1cd0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:16.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cd01000d0 0x7f1cd0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 -- 172.21.15.19:0/4108698780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd019d500 con 0x7f1cd0100a70 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1ccf7fe640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd0106ba0 0x7f1cd019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cce7fc640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0100a70 0x7f1cd0198a90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400d950 tx=0x7f1cc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cb7fff640 1 -- 172.21.15.19:0/4108698780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4017070 con 0x7f1cd0100a70 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cb7fff640 1 -- 172.21.15.19:0/4108698780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc4009c30 con 0x7f1cd0100a70 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd019d7f0 con 0x7f1cd0100a70 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.313+0000 7f1cb7fff640 1 -- 172.21.15.19:0/4108698780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400f6b0 con 0x7f1cd0100a70 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.314+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd01a50d0 con 0x7f1cd0100a70 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.315+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd0100530 con 0x7f1cd0100a70 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.316+0000 7f1cb7fff640 1 -- 172.21.15.19:0/4108698780 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1cc40109d0 con 0x7f1cd0100a70 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.316+0000 7f1cb7fff640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1cb80a32c0 0x7f1cb80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.317+0000 7f1cceffd640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1cb80a32c0 0x7f1cb80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.317+0000 7f1cb7fff640 1 -- 172.21.15.19:0/4108698780 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f1cc40cf030 con 0x7f1cd0100a70 2024-09-19T04:04:16.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.317+0000 7f1cceffd640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1cb80a32c0 0x7f1cb80a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009230 tx=0x7f1cc00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:16.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.320+0000 7f1cb7fff640 1 -- 172.21.15.19:0/4108698780 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1cc4009de0 con 0x7f1cd0100a70 2024-09-19T04:04:16.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:15 smithi111 ceph-mon[35365]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:16.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.529+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cd01998b0 con 0x7f1cd0100a70 2024-09-19T04:04:16.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.530+0000 7f1cb7fff640 1 -- 172.21.15.19:0/4108698780 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f1cc409c270 con 0x7f1cd0100a70 2024-09-19T04:04:16.532 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:16.532 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"btime":"2024-09-19T04:04:15:962638+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T04:04:16.532 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-09-19T04:04:16.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1cb80a32c0 msgr2=0x7f1cb80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1cb80a32c0 0x7f1cb80a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009230 tx=0x7f1cc00058d0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0100a70 msgr2=0x7f1cd0198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0100a70 0x7f1cd0198a90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400d950 tx=0x7f1cc400de10 comp rx=0 tx=0).stop 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 shutdown_connections 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1cb80a32c0 0x7f1cb80a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd0106ba0 0x7f1cd019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cd0100a70 0x7f1cd0198a90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 --2- 172.21.15.19:0/4108698780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cd01000d0 0x7f1cd0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.532+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 >> 172.21.15.19:0/4108698780 conn(0x7f1cd0077210 msgr2=0x7f1cd0104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.533+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 shutdown_connections 2024-09-19T04:04:16.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:16.533+0000 7f1cd53ce640 1 -- 172.21.15.19:0/4108698780 wait complete. 2024-09-19T04:04:17.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] up:boot 2024-09-19T04:04:17.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: fsmap 7 up:standby 2024-09-19T04:04:17.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:04:17.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:17.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:17.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:17.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: Deploying daemon mds.h on smithi111 2024-09-19T04:04:17.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:16 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/4108698780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:17.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] up:boot 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: fsmap 7 up:standby 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: Deploying daemon mds.h on smithi111 2024-09-19T04:04:17.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:16 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4108698780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:17.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] up:boot 2024-09-19T04:04:17.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: fsmap 7 up:standby 2024-09-19T04:04:17.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:04:17.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:17.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:17.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:17.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:17.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: Deploying daemon mds.h on smithi111 2024-09-19T04:04:17.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:16 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/4108698780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:18.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:18 smithi111 ceph-mon[35365]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:18.405 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 systemd[1]: Starting Ceph mds.h for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:18.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:18 smithi019 ceph-mon[30938]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:18 smithi196 ceph-mon[35731]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:18.743 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 podman[64976]: 2024-09-19 04:04:18.327614157 +0000 UTC m=+0.016089834 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:18.744 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 podman[64976]: 2024-09-19 04:04:18.464775211 +0000 UTC m=+0.153250886 container create affc28ae2aa2fefd35a37377bcf1c0843a282874ccbe9c8fb4a781e11809b9b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:04:18.744 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 podman[64976]: 2024-09-19 04:04:18.742953231 +0000 UTC m=+0.431428908 container init affc28ae2aa2fefd35a37377bcf1c0843a282874ccbe9c8fb4a781e11809b9b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T04:04:19.155 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 podman[64976]: 2024-09-19 04:04:18.745704857 +0000 UTC m=+0.434180536 container start affc28ae2aa2fefd35a37377bcf1c0843a282874ccbe9c8fb4a781e11809b9b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T04:04:19.155 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 ceph-mds[64995]: -- 172.21.15.111:0/4184659641 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d28f3a9860 con 0x55d28f386800 2024-09-19T04:04:19.155 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h[64991]: starting mds.h at 2024-09-19T04:04:19.155 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 bash[64976]: affc28ae2aa2fefd35a37377bcf1c0843a282874ccbe9c8fb4a781e11809b9b9 2024-09-19T04:04:19.155 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:04:18 smithi111 systemd[1]: Started Ceph mds.h for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:19.167 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 -- 172.21.15.19:0/1163804839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4106f10 msgr2=0x7fb1d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/1163804839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4106f10 0x7fb1d410ea90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009ca0 tx=0x7fb1cc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:04:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 -- 172.21.15.19:0/1163804839 shutdown_connections 2024-09-19T04:04:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/1163804839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4106f10 0x7fb1d410ea90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/1163804839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d4102790 0x7fb1d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/1163804839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d4101de0 0x7fb1d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 -- 172.21.15.19:0/1163804839 >> 172.21.15.19:0/1163804839 conn(0x7fb1d40fd8c0 msgr2=0x7fb1d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 -- 172.21.15.19:0/1163804839 shutdown_connections 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.960+0000 7fb1dc2be640 1 -- 172.21.15.19:0/1163804839 wait complete. 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.961+0000 7fb1dc2be640 1 Processor -- start 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.961+0000 7fb1dc2be640 1 -- start start 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.961+0000 7fb1dc2be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d4101de0 0x7fb1d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.961+0000 7fb1dc2be640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4102790 0x7fb1d4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.961+0000 7fb1dc2be640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d4106f10 0x7fb1d4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.961+0000 7fb1dc2be640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d4111f70 con 0x7fb1d4102790 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.961+0000 7fb1dc2be640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d4111df0 con 0x7fb1d4106f10 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.961+0000 7fb1dc2be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d41120f0 con 0x7fb1d4101de0 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4102790 0x7fb1d4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4102790 0x7fb1d4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45760/0 (socket says 172.21.15.19:45760) 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 -- 172.21.15.19:0/22879161 learned_addr learned my addr 172.21.15.19:0/22879161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1da05a640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d4101de0 0x7fb1d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1da85b640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d4106f10 0x7fb1d4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 -- 172.21.15.19:0/22879161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d4101de0 msgr2=0x7fb1d40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d4101de0 0x7fb1d40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 -- 172.21.15.19:0/22879161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d4106f10 msgr2=0x7fb1d4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d4106f10 0x7fb1d4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 -- 172.21.15.19:0/22879161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d41a67f0 con 0x7fb1d4102790 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1da85b640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d4106f10 0x7fb1d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1d9859640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4102790 0x7fb1d4074a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d770 tx=0x7fb1c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1cb7fe640 1 -- 172.21.15.19:0/22879161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4017070 con 0x7fb1d4102790 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1cb7fe640 1 -- 172.21.15.19:0/22879161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c4005500 con 0x7fb1d4102790 2024-09-19T04:04:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d41a6ae0 con 0x7fb1d4102790 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1cb7fe640 1 -- 172.21.15.19:0/22879161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4004e00 con 0x7fb1d4102790 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.962+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d41a7020 con 0x7fb1d4102790 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.963+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d41021c0 con 0x7fb1d4102790 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.964+0000 7fb1cb7fe640 1 -- 172.21.15.19:0/22879161 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb1c4004fa0 con 0x7fb1d4102790 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.965+0000 7fb1cb7fe640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb1c00a32c0 0x7fb1c00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.965+0000 7fb1da05a640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb1c00a32c0 0x7fb1c00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.965+0000 7fb1cb7fe640 1 -- 172.21.15.19:0/22879161 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fb1c40cee80 con 0x7fb1d4102790 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.966+0000 7fb1da05a640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb1c00a32c0 0x7fb1c00a5780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0005e10 tx=0x7fb1d0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:19.969+0000 7fb1cb7fe640 1 -- 172.21.15.19:0/22879161 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb1c409c040 con 0x7fb1d4102790 2024-09-19T04:04:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:20.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: Deploying daemon mds.i on smithi196 2024-09-19T04:04:20.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] up:boot 2024-09-19T04:04:20.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: fsmap 8 up:standby 2024-09-19T04:04:20.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:19 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:04:20.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:20.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:20.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:20.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:20.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: Deploying daemon mds.i on smithi196 2024-09-19T04:04:20.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] up:boot 2024-09-19T04:04:20.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: fsmap 8 up:standby 2024-09-19T04:04:20.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:19 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:04:20.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.182+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1d4109b60 con 0x7fb1d4102790 2024-09-19T04:04:20.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.182+0000 7fb1cb7fe640 1 -- 172.21.15.19:0/22879161 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7fb1c409b940 con 0x7fb1d4102790 2024-09-19T04:04:20.184 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:20.184 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"btime":"2024-09-19T04:04:19:121244+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T04:04:20.184 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-09-19T04:04:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb1c00a32c0 msgr2=0x7fb1c00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb1c00a32c0 0x7fb1c00a5780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0005e10 tx=0x7fb1d0005da0 comp rx=0 tx=0).stop 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4102790 msgr2=0x7fb1d4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4102790 0x7fb1d4074a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d770 tx=0x7fb1c400dc30 comp rx=0 tx=0).stop 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 shutdown_connections 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb1c00a32c0 0x7fb1c00a5780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1d4106f10 0x7fb1d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d4102790 0x7fb1d4074a30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 --2- 172.21.15.19:0/22879161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d4101de0 0x7fb1d40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.184+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 >> 172.21.15.19:0/22879161 conn(0x7fb1d40fd8c0 msgr2=0x7fb1d40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.185+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 shutdown_connections 2024-09-19T04:04:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:20.185+0000 7fb1dc2be640 1 -- 172.21.15.19:0/22879161 wait complete. 2024-09-19T04:04:20.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:20.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:20.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:20.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:20.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:20.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: Deploying daemon mds.i on smithi196 2024-09-19T04:04:20.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] up:boot 2024-09-19T04:04:20.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: fsmap 8 up:standby 2024-09-19T04:04:20.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:19 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:04:20.892 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:20 smithi196 systemd[1]: Starting Ceph mds.i for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:21.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:20 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/22879161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:21.161 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:20 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/22879161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:21.161 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:21 smithi196 podman[65771]: 2024-09-19 04:04:20.9101126 +0000 UTC m=+0.016371825 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:21.161 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:21 smithi196 podman[65771]: 2024-09-19 04:04:21.033542575 +0000 UTC m=+0.139801832 container create c1594205ad50ad907842433c69a84ddb218644c3418e87749c509c95c05dec3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T04:04:21.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:20 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/22879161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:21.414 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:21 smithi196 podman[65771]: 2024-09-19 04:04:21.304245758 +0000 UTC m=+0.410504973 container init c1594205ad50ad907842433c69a84ddb218644c3418e87749c509c95c05dec3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:04:21.414 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:21 smithi196 podman[65771]: 2024-09-19 04:04:21.307120442 +0000 UTC m=+0.413379658 container start c1594205ad50ad907842433c69a84ddb218644c3418e87749c509c95c05dec3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T04:04:21.415 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-mds[65790]: -- 172.21.15.196:0/2498775462 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4aa47f860 con 0x55b4aa45d000 2024-09-19T04:04:21.415 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i[65786]: starting mds.i at 2024-09-19T04:04:21.415 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:21 smithi196 bash[65771]: c1594205ad50ad907842433c69a84ddb218644c3418e87749c509c95c05dec3d 2024-09-19T04:04:21.708 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:04:21 smithi196 systemd[1]: Started Ceph mds.i for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:21 smithi111 ceph-mon[35365]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:22.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:21 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:21 smithi019 ceph-mon[30938]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:21 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:22.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-mon[35731]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:22.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:22.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:22.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:22.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:21 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:22.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:23.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:22 smithi019 ceph-mon[30938]: Deploying daemon mds.g on smithi019 2024-09-19T04:04:23.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:22 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] up:boot 2024-09-19T04:04:23.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:22 smithi019 ceph-mon[30938]: fsmap 9 up:standby 2024-09-19T04:04:23.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:22 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:04:23.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:22 smithi196 ceph-mon[35731]: Deploying daemon mds.g on smithi019 2024-09-19T04:04:23.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:22 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] up:boot 2024-09-19T04:04:23.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:22 smithi196 ceph-mon[35731]: fsmap 9 up:standby 2024-09-19T04:04:23.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:22 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:04:23.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:22 smithi111 ceph-mon[35365]: Deploying daemon mds.g on smithi019 2024-09-19T04:04:23.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:22 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] up:boot 2024-09-19T04:04:23.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:22 smithi111 ceph-mon[35365]: fsmap 9 up:standby 2024-09-19T04:04:23.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:22 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:04:23.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/580151541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 msgr2=0x7f5eb01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:23.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/580151541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 0x7f5eb01067e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009ca0 tx=0x7f5e9802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/580151541 shutdown_connections 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/580151541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5eb0106f10 0x7f5eb010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/580151541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 0x7f5eb01067e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/580151541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb0101de0 0x7f5eb01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/580151541 >> 172.21.15.19:0/580151541 conn(0x7f5eb00fd8c0 msgr2=0x7f5eb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/580151541 shutdown_connections 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.702+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/580151541 wait complete. 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.703+0000 7f5eb4a1a640 1 Processor -- start 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.703+0000 7f5eb4a1a640 1 -- start start 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.703+0000 7f5eb4a1a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5eb0101de0 0x7f5eb00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.703+0000 7f5eb4a1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 0x7f5eb0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eb4a1a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb0106f10 0x7f5eb0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eb4a1a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0111fb0 con 0x7f5eb0102790 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eb4a1a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0111e30 con 0x7f5eb0101de0 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eb4a1a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0112130 con 0x7f5eb0106f10 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 0x7f5eb0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 0x7f5eb0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43336/0 (socket says 172.21.15.19:43336) 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 -- 172.21.15.19:0/3179941214 learned_addr learned my addr 172.21.15.19:0/3179941214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eaed9d640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb0106f10 0x7f5eb0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eae59c640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5eb0101de0 0x7f5eb00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 -- 172.21.15.19:0/3179941214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb0106f10 msgr2=0x7f5eb0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb0106f10 0x7f5eb0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 -- 172.21.15.19:0/3179941214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5eb0101de0 msgr2=0x7f5eb00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5eb0101de0 0x7f5eb00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 -- 172.21.15.19:0/3179941214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb01a6970 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eadd9b640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 0x7f5eb0074a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5e9802ade0 tx=0x7f5e980319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5ea77fe640 1 -- 172.21.15.19:0/3179941214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9802fa20 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5ea77fe640 1 -- 172.21.15.19:0/3179941214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e9802fbc0 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb01a6c00 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5ea77fe640 1 -- 172.21.15.19:0/3179941214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980466e0 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.704+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb01a7190 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.706+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb01021c0 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.706+0000 7f5ea77fe640 1 -- 172.21.15.19:0/3179941214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5e98046940 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.707+0000 7f5ea77fe640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e9c0a32c0 0x7f5e9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.707+0000 7f5eae59c640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e9c0a32c0 0x7f5e9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.708+0000 7f5ea77fe640 1 -- 172.21.15.19:0/3179941214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f5e980f0940 con 0x7f5eb0102790 2024-09-19T04:04:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.708+0000 7f5eae59c640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e9c0a32c0 0x7f5e9c0a5780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009bd0 tx=0x7f5ea0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:23.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.711+0000 7f5ea77fe640 1 -- 172.21.15.19:0/3179941214 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5e980bdb00 con 0x7f5eb0102790 2024-09-19T04:04:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.931+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eb010a150 con 0x7f5eb0102790 2024-09-19T04:04:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.931+0000 7f5ea77fe640 1 -- 172.21.15.19:0/3179941214 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7f5e980bd400 con 0x7f5eb0102790 2024-09-19T04:04:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:23.933 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"btime":"2024-09-19T04:04:21:891901+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:04:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-09-19T04:04:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e9c0a32c0 msgr2=0x7f5e9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e9c0a32c0 0x7f5e9c0a5780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009bd0 tx=0x7f5ea0005f90 comp rx=0 tx=0).stop 2024-09-19T04:04:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 msgr2=0x7f5eb0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 0x7f5eb0074a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5e9802ade0 tx=0x7f5e980319f0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 shutdown_connections 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5e9c0a32c0 0x7f5e9c0a5780 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb0106f10 0x7f5eb0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5eb0102790 0x7f5eb0074a30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 --2- 172.21.15.19:0/3179941214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5eb0101de0 0x7f5eb00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 >> 172.21.15.19:0/3179941214 conn(0x7f5eb00fd8c0 msgr2=0x7f5eb00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 shutdown_connections 2024-09-19T04:04:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:23.934+0000 7f5eb4a1a640 1 -- 172.21.15.19:0/3179941214 wait complete. 2024-09-19T04:04:24.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:23 smithi019 ceph-mon[30938]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:24.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:23 smithi196 ceph-mon[35731]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:24.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:23 smithi111 ceph-mon[35365]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:25.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:24 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3179941214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:25.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:24 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:25.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:24 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3179941214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:25.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:25.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:25.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:24 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:25.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:24 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3179941214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:25.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:25.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:25.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:25.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:24 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:26.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:25 smithi019 ceph-mon[30938]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:26.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:25 smithi019 ceph-mon[30938]: Deploying daemon mds.a on smithi019 2024-09-19T04:04:26.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:25 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.19:6838/821554910,v1:172.21.15.19:6839/821554910] up:boot 2024-09-19T04:04:26.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:25 smithi019 ceph-mon[30938]: fsmap 10 up:standby 2024-09-19T04:04:26.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:25 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:04:26.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:25 smithi196 ceph-mon[35731]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:26.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:25 smithi196 ceph-mon[35731]: Deploying daemon mds.a on smithi019 2024-09-19T04:04:26.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:25 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.19:6838/821554910,v1:172.21.15.19:6839/821554910] up:boot 2024-09-19T04:04:26.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:25 smithi196 ceph-mon[35731]: fsmap 10 up:standby 2024-09-19T04:04:26.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:25 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:04:26.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:25 smithi111 ceph-mon[35365]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:26.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:25 smithi111 ceph-mon[35365]: Deploying daemon mds.a on smithi019 2024-09-19T04:04:26.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:25 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.19:6838/821554910,v1:172.21.15.19:6839/821554910] up:boot 2024-09-19T04:04:26.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:25 smithi111 ceph-mon[35365]: fsmap 10 up:standby 2024-09-19T04:04:26.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:25 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:04:26.612 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:26 smithi019 systemd[1]: Starting Ceph mds.a for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:26.901 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:26.912 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:26 smithi019 podman[71558]: 2024-09-19 04:04:26.679680102 +0000 UTC m=+0.016791287 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:26.912 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:26 smithi019 podman[71558]: 2024-09-19 04:04:26.796245458 +0000 UTC m=+0.133356673 container create d719fb1c5e70bda387eb33d34edd17cb0a42eb535e13d4f1c00a426fb70e485f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T04:04:27.181 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:27 smithi019 podman[71558]: 2024-09-19 04:04:27.006966653 +0000 UTC m=+0.344077839 container init d719fb1c5e70bda387eb33d34edd17cb0a42eb535e13d4f1c00a426fb70e485f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T04:04:27.181 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:27 smithi019 podman[71558]: 2024-09-19 04:04:27.009905321 +0000 UTC m=+0.347016498 container start d719fb1c5e70bda387eb33d34edd17cb0a42eb535e13d4f1c00a426fb70e485f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T04:04:27.181 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:27 smithi019 ceph-mds[71595]: -- 172.21.15.19:0/1847369784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594965a3860 con 0x559496580800 2024-09-19T04:04:27.182 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:27 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a[71574]: starting mds.a at 2024-09-19T04:04:27.182 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:27 smithi019 bash[71558]: d719fb1c5e70bda387eb33d34edd17cb0a42eb535e13d4f1c00a426fb70e485f 2024-09-19T04:04:27.182 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:04:27 smithi019 systemd[1]: Started Ceph mds.a for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:27.183 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:27.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 -- 172.21.15.19:0/2506741093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 msgr2=0x7f19cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:27.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 --2- 172.21.15.19:0/2506741093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 0x7f19cc1029b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009900 tx=0x7f19c002a480 comp rx=0 tx=0).stop 2024-09-19T04:04:27.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 -- 172.21.15.19:0/2506741093 shutdown_connections 2024-09-19T04:04:27.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 --2- 172.21.15.19:0/2506741093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc102ef0 0x7f19cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 --2- 172.21.15.19:0/2506741093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 0x7f19cc1029b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 --2- 172.21.15.19:0/2506741093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc100e20 0x7f19cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 -- 172.21.15.19:0/2506741093 >> 172.21.15.19:0/2506741093 conn(0x7f19cc0fc7d0 msgr2=0x7f19cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 -- 172.21.15.19:0/2506741093 shutdown_connections 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.635+0000 7f19d0e84640 1 -- 172.21.15.19:0/2506741093 wait complete. 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19d0e84640 1 Processor -- start 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19d0e84640 1 -- start start 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19d0e84640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc100e20 0x7f19cc199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19d0e84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 0x7f19cc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19d0e84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc102ef0 0x7f19cc1a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19d0e84640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc110f80 con 0x7f19cc102570 2024-09-19T04:04:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19d0e84640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc110e00 con 0x7f19cc100e20 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19d0e84640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc111100 con 0x7f19cc102ef0 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19c9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 0x7f19cc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19c9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 0x7f19cc1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43396/0 (socket says 172.21.15.19:43396) 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19c9d9b640 1 -- 172.21.15.19:0/4228349748 learned_addr learned my addr 172.21.15.19:0/4228349748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.636+0000 7f19ca59c640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc100e20 0x7f19cc199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19cad9d640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc102ef0 0x7f19cc1a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19c9d9b640 1 -- 172.21.15.19:0/4228349748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc102ef0 msgr2=0x7f19cc1a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19c9d9b640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc102ef0 0x7f19cc1a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19c9d9b640 1 -- 172.21.15.19:0/4228349748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc100e20 msgr2=0x7f19cc199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19c9d9b640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc100e20 0x7f19cc199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19c9d9b640 1 -- 172.21.15.19:0/4228349748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc1a4950 con 0x7f19cc102570 2024-09-19T04:04:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19ca59c640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc100e20 0x7f19cc199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19c9d9b640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 0x7f19cc1998d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f19c0006010 tx=0x7f19c0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19bb7fe640 1 -- 172.21.15.19:0/4228349748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c003f070 con 0x7f19cc102570 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc1a4b20 con 0x7f19cc102570 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19bb7fe640 1 -- 172.21.15.19:0/4228349748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19c0031c30 con 0x7f19cc102570 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19bb7fe640 1 -- 172.21.15.19:0/4228349748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0035030 con 0x7f19cc102570 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.637+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc1a4f70 con 0x7f19cc102570 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.638+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19cc101200 con 0x7f19cc102570 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.639+0000 7f19bb7fe640 1 -- 172.21.15.19:0/4228349748 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f19c0042070 con 0x7f19cc102570 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.640+0000 7f19bb7fe640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19b40a3240 0x7f19b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.640+0000 7f19ca59c640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19b40a3240 0x7f19b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.640+0000 7f19bb7fe640 1 -- 172.21.15.19:0/4228349748 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f19c00efd70 con 0x7f19cc102570 2024-09-19T04:04:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.640+0000 7f19ca59c640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19b40a3240 0x7f19b40a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00bfd0 tx=0x7f19bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.644+0000 7f19bb7fe640 1 -- 172.21.15.19:0/4228349748 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f19c0031de0 con 0x7f19cc102570 2024-09-19T04:04:27.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.861+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19cc109090 con 0x7f19cc102570 2024-09-19T04:04:27.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.862+0000 7f19bb7fe640 1 -- 172.21.15.19:0/4228349748 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8878 (secure 0 0 0) 0x7f19c00bce90 con 0x7f19cc102570 2024-09-19T04:04:27.863 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:27.863 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"btime":"2024-09-19T04:04:24:904678+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T04:04:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-09-19T04:04:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19b40a3240 msgr2=0x7f19b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19b40a3240 0x7f19b40a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00bfd0 tx=0x7f19bc00b390 comp rx=0 tx=0).stop 2024-09-19T04:04:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 msgr2=0x7f19cc1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 0x7f19cc1998d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f19c0006010 tx=0x7f19c0003c10 comp rx=0 tx=0).stop 2024-09-19T04:04:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 shutdown_connections 2024-09-19T04:04:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19b40a3240 0x7f19b40a5700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc102ef0 0x7f19cc1a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19cc102570 0x7f19cc1998d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 --2- 172.21.15.19:0/4228349748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19cc100e20 0x7f19cc199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 >> 172.21.15.19:0/4228349748 conn(0x7f19cc0fc7d0 msgr2=0x7f19cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.864+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 shutdown_connections 2024-09-19T04:04:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:27.865+0000 7f19d0e84640 1 -- 172.21.15.19:0/4228349748 wait complete. 2024-09-19T04:04:28.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:28.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:28.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:28.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:28.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: Deploying daemon mds.b on smithi111 2024-09-19T04:04:28.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4228349748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:28.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.19:6840/733319067,v1:172.21.15.19:6841/733319067] up:boot 2024-09-19T04:04:28.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: fsmap 11 up:standby 2024-09-19T04:04:28.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:28 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:04:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: Deploying daemon mds.b on smithi111 2024-09-19T04:04:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/4228349748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.19:6840/733319067,v1:172.21.15.19:6841/733319067] up:boot 2024-09-19T04:04:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: fsmap 11 up:standby 2024-09-19T04:04:28.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:28 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:04:28.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:28.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: Deploying daemon mds.b on smithi111 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/4228349748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.19:6840/733319067,v1:172.21.15.19:6841/733319067] up:boot 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: fsmap 11 up:standby 2024-09-19T04:04:28.623 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:28 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:04:29.405 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 systemd[1]: Starting Ceph mds.b for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:04:29.732 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 podman[65288]: 2024-09-19 04:04:29.344229937 +0000 UTC m=+0.016129440 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:04:29.733 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 podman[65288]: 2024-09-19 04:04:29.470834986 +0000 UTC m=+0.142734488 container create f4d3c6ec631bff6d0906979b91e518f2fb99413ad352bd4933aa318866765b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-09-19T04:04:29.733 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 podman[65288]: 2024-09-19 04:04:29.731876 +0000 UTC m=+0.403775502 container init f4d3c6ec631bff6d0906979b91e518f2fb99413ad352bd4933aa318866765b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T04:04:30.155 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 podman[65288]: 2024-09-19 04:04:29.734777284 +0000 UTC m=+0.406676791 container start f4d3c6ec631bff6d0906979b91e518f2fb99413ad352bd4933aa318866765b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T04:04:30.155 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 ceph-mds[65308]: -- 172.21.15.111:0/2935220215 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56519d055860 con 0x56519d033400 2024-09-19T04:04:30.155 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: starting mds.b at 2024-09-19T04:04:30.155 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 bash[65288]: f4d3c6ec631bff6d0906979b91e518f2fb99413ad352bd4933aa318866765b5d 2024-09-19T04:04:30.155 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:29 smithi111 systemd[1]: Started Ceph mds.b for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:04:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:30 smithi019 ceph-mon[30938]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:30 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:04:30.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:30 smithi196 ceph-mon[35731]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:30 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:04:30.621 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:30 smithi111 ceph-mon[35365]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:30.621 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:30.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:30 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:04:31.145 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:31.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:31 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] up:boot 2024-09-19T04:04:31.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:31 smithi019 ceph-mon[30938]: fsmap 12 up:standby 2024-09-19T04:04:31.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:04:31.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:31 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:31 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] up:boot 2024-09-19T04:04:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:31 smithi196 ceph-mon[35731]: fsmap 12 up:standby 2024-09-19T04:04:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:04:31.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:31 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.629+0000 7f527022e640 1 -- 172.21.15.19:0/768529505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52681008d0 msgr2=0x7f5268100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.629+0000 7f527022e640 1 --2- 172.21.15.19:0/768529505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52681008d0 0x7f5268100cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5264009900 tx=0x7f526402a410 comp rx=0 tx=0).stop 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.629+0000 7f527022e640 1 -- 172.21.15.19:0/768529505 shutdown_connections 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.629+0000 7f527022e640 1 --2- 172.21.15.19:0/768529505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f526810e480 0x7f5268110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.629+0000 7f527022e640 1 --2- 172.21.15.19:0/768529505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5268101280 0x7f526810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.629+0000 7f527022e640 1 --2- 172.21.15.19:0/768529505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52681008d0 0x7f5268100cb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.629+0000 7f527022e640 1 -- 172.21.15.19:0/768529505 >> 172.21.15.19:0/768529505 conn(0x7f52680fc7a0 msgr2=0x7f52680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 -- 172.21.15.19:0/768529505 shutdown_connections 2024-09-19T04:04:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 -- 172.21.15.19:0/768529505 wait complete. 2024-09-19T04:04:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 Processor -- start 2024-09-19T04:04:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 -- start start 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52681008d0 0x7f526819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5268101280 0x7f526819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f526810e480 0x7f526819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5268114200 con 0x7f526810e480 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5268114080 con 0x7f5268101280 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.630+0000 7f527022e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5268114380 con 0x7f52681008d0 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f526810e480 0x7f526819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f526810e480 0x7f526819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47972/0 (socket says 172.21.15.19:47972) 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 -- 172.21.15.19:0/1472875670 learned_addr learned my addr 172.21.15.19:0/1472875670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526dfca640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52681008d0 0x7f526819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526d7c9640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5268101280 0x7f526819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 -- 172.21.15.19:0/1472875670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52681008d0 msgr2=0x7f526819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52681008d0 0x7f526819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 -- 172.21.15.19:0/1472875670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5268101280 msgr2=0x7f526819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5268101280 0x7f526819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 -- 172.21.15.19:0/1472875670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526819f6f0 con 0x7f526810e480 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526d7c9640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5268101280 0x7f526819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f526e7cb640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f526810e480 0x7f526819f010 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5260009fd0 tx=0x7f526000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f525effd640 1 -- 172.21.15.19:0/1472875670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260004d20 con 0x7f526810e480 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f525effd640 1 -- 172.21.15.19:0/1472875670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5260005020 con 0x7f526810e480 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526819f9e0 con 0x7f526810e480 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.631+0000 7f525effd640 1 -- 172.21.15.19:0/1472875670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600116d0 con 0x7f526810e480 2024-09-19T04:04:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.632+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52681a72c0 con 0x7f526810e480 2024-09-19T04:04:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.633+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5268100cb0 con 0x7f526810e480 2024-09-19T04:04:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.634+0000 7f525effd640 1 -- 172.21.15.19:0/1472875670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f52600051d0 con 0x7f526810e480 2024-09-19T04:04:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.635+0000 7f525effd640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f52540a3240 0x7f52540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.635+0000 7f526dfca640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f52540a3240 0x7f52540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.635+0000 7f525effd640 1 -- 172.21.15.19:0/1472875670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f52600cfab0 con 0x7f526810e480 2024-09-19T04:04:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.635+0000 7f526dfca640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f52540a3240 0x7f52540a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5264000c00 tx=0x7f5264005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.638+0000 7f525effd640 1 -- 172.21.15.19:0/1472875670 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5260016030 con 0x7f526810e480 2024-09-19T04:04:31.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:31 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] up:boot 2024-09-19T04:04:31.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:31 smithi111 ceph-mon[35365]: fsmap 12 up:standby 2024-09-19T04:04:31.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:04:31.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:31 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:31.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.865+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5268109610 con 0x7f526810e480 2024-09-19T04:04:31.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.866+0000 7f525effd640 1 -- 172.21.15.19:0/1472875670 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10527 (secure 0 0 0) 0x7f52600bc030 con 0x7f526810e480 2024-09-19T04:04:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:31.868 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"btime":"2024-09-19T04:04:30:166910+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T04:04:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f52540a3240 msgr2=0x7f52540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f52540a3240 0x7f52540a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5264000c00 tx=0x7f5264005b50 comp rx=0 tx=0).stop 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f526810e480 msgr2=0x7f526819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f526810e480 0x7f526819f010 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5260009fd0 tx=0x7f526000ee60 comp rx=0 tx=0).stop 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 shutdown_connections 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f52540a3240 0x7f52540a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f526810e480 0x7f526819f010 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5268101280 0x7f526819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 --2- 172.21.15.19:0/1472875670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52681008d0 0x7f526819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.868+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 >> 172.21.15.19:0/1472875670 conn(0x7f52680fc7a0 msgr2=0x7f52680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.869+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 shutdown_connections 2024-09-19T04:04:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:31.869+0000 7f527022e640 1 -- 172.21.15.19:0/1472875670 wait complete. 2024-09-19T04:04:32.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:32.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1472875670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:04:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:32 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:04:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1472875670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:04:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:32 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:04:32.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T04:04:32.649 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:32.649 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.649 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.649 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.649 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.650 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1472875670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:32.650 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.650 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.650 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:32.650 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:04:32.650 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:32.650 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:32 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:04:33.381 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 -- 172.21.15.19:0/3359124350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c112b90 msgr2=0x7f677c114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3359124350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c112b90 0x7f677c114f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6774009900 tx=0x7f677402a480 comp rx=0 tx=0).stop 2024-09-19T04:04:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 -- 172.21.15.19:0/3359124350 shutdown_connections 2024-09-19T04:04:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3359124350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c112b90 0x7f677c114f80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3359124350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f677c074eb0 0x7f677c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3359124350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f677c074590 0x7f677c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 -- 172.21.15.19:0/3359124350 >> 172.21.15.19:0/3359124350 conn(0x7f677c0fd8c0 msgr2=0x7f677c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 -- 172.21.15.19:0/3359124350 shutdown_connections 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.884+0000 7f6783e2e640 1 -- 172.21.15.19:0/3359124350 wait complete. 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6783e2e640 1 Processor -- start 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6783e2e640 1 -- start start 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6783e2e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f677c074590 0x7f677c1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6783e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c074eb0 0x7f677c1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6783e2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f677c112b90 0x7f677c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6783e2e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1185c0 con 0x7f677c074eb0 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6783e2e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f677c118440 con 0x7f677c074590 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6783e2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f677c118740 con 0x7f677c112b90 2024-09-19T04:04:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67813c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c074eb0 0x7f677c1a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67813c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c074eb0 0x7f677c1a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47998/0 (socket says 172.21.15.19:47998) 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67813c9640 1 -- 172.21.15.19:0/3467557515 learned_addr learned my addr 172.21.15.19:0/3467557515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6781bca640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f677c074590 0x7f677c1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67823cb640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f677c112b90 0x7f677c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67813c9640 1 -- 172.21.15.19:0/3467557515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f677c112b90 msgr2=0x7f677c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67813c9640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f677c112b90 0x7f677c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67813c9640 1 -- 172.21.15.19:0/3467557515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f677c074590 msgr2=0x7f677c1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67813c9640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f677c074590 0x7f677c1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f67813c9640 1 -- 172.21.15.19:0/3467557515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c19f010 con 0x7f677c074eb0 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.885+0000 7f6781bca640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f677c074590 0x7f677c1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.886+0000 7f67813c9640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c074eb0 0x7f677c1a4c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f677000c890 tx=0x7f677000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.886+0000 7f677affd640 1 -- 172.21.15.19:0/3467557515 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770004220 con 0x7f677c074eb0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.886+0000 7f677affd640 1 -- 172.21.15.19:0/3467557515 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67700043c0 con 0x7f677c074eb0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.886+0000 7f677affd640 1 -- 172.21.15.19:0/3467557515 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67700049e0 con 0x7f677c074eb0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.886+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c19f300 con 0x7f677c074eb0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.886+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1ab5f0 con 0x7f677c074eb0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.887+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c074970 con 0x7f677c074eb0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.888+0000 7f677affd640 1 -- 172.21.15.19:0/3467557515 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f67700079b0 con 0x7f677c074eb0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.889+0000 7f677affd640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f67640a3240 0x7f67640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.889+0000 7f6781bca640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f67640a3240 0x7f67640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.889+0000 7f677affd640 1 -- 172.21.15.19:0/3467557515 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f67700cf5a0 con 0x7f677c074eb0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.889+0000 7f6781bca640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f67640a3240 0x7f67640a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f677c19fa20 tx=0x7f676c009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:33.892+0000 7f677affd640 1 -- 172.21.15.19:0/3467557515 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6770014030 con 0x7f677c074eb0 2024-09-19T04:04:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.080+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f677c19fe10 con 0x7f677c074eb0 2024-09-19T04:04:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.080+0000 7f677affd640 1 -- 172.21.15.19:0/3467557515 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7f677c19fe10 con 0x7f677c074eb0 2024-09-19T04:04:34.083 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:34.083 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-09-19T04:04:34.083 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:04:34.083 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-09-19T04:04:34.083 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-09-19T04:04:34.083 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:04:34.083 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-09-19T04:04:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f67640a3240 msgr2=0x7f67640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f67640a3240 0x7f67640a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f677c19fa20 tx=0x7f676c009570 comp rx=0 tx=0).stop 2024-09-19T04:04:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c074eb0 msgr2=0x7f677c1a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c074eb0 0x7f677c1a4c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f677000c890 tx=0x7f677000cd50 comp rx=0 tx=0).stop 2024-09-19T04:04:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 shutdown_connections 2024-09-19T04:04:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f67640a3240 0x7f67640a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f677c112b90 0x7f677c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:34.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677c074eb0 0x7f677c1a4c60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:34.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 --2- 172.21.15.19:0/3467557515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f677c074590 0x7f677c1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:34.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 >> 172.21.15.19:0/3467557515 conn(0x7f677c0fd8c0 msgr2=0x7f677c0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:34.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 shutdown_connections 2024-09-19T04:04:34.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:34.083+0000 7f6783e2e640 1 -- 172.21.15.19:0/3467557515 wait complete. 2024-09-19T04:04:34.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:34.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3467557515' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:04:34.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:34.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:04:34.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:34.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3467557515' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:04:34.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:34.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:04:34.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:34.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3467557515' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T04:04:34.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:04:34.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:04:34.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:04:34.709 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:04:34.709 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:04:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 -- 172.21.15.19:0/3559227558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102ef0 msgr2=0x7f709010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 --2- 172.21.15.19:0/3559227558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102ef0 0x7f709010da40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7088009ca0 tx=0x7f708802a950 comp rx=0 tx=0).stop 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 -- 172.21.15.19:0/3559227558 shutdown_connections 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 --2- 172.21.15.19:0/3559227558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102ef0 0x7f709010da40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 --2- 172.21.15.19:0/3559227558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7090102570 0x7f70901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 --2- 172.21.15.19:0/3559227558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7090100e20 0x7f7090101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 -- 172.21.15.19:0/3559227558 >> 172.21.15.19:0/3559227558 conn(0x7f70900fc7d0 msgr2=0x7f70900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 -- 172.21.15.19:0/3559227558 shutdown_connections 2024-09-19T04:04:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.395+0000 7f7095da6640 1 -- 172.21.15.19:0/3559227558 wait complete. 2024-09-19T04:04:35.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f7095da6640 1 Processor -- start 2024-09-19T04:04:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f7095da6640 1 -- start start 2024-09-19T04:04:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f7095da6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7090100e20 0x7f70901984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f7095da6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102570 0x7f7090198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f7095da6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7090102ef0 0x7f709019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f7095da6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7090110fa0 con 0x7f7090102570 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f7095da6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7090110e20 con 0x7f7090100e20 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f7095da6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7090111120 con 0x7f7090102ef0 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.396+0000 7f708effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102570 0x7f7090198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102570 0x7f7090198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48008/0 (socket says 172.21.15.19:48008) 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708effd640 1 -- 172.21.15.19:0/3299569290 learned_addr learned my addr 172.21.15.19:0/3299569290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708f7fe640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7090100e20 0x7f70901984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708effd640 1 -- 172.21.15.19:0/3299569290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7090102ef0 msgr2=0x7f709019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708ffff640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7090102ef0 0x7f709019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708effd640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7090102ef0 0x7f709019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708effd640 1 -- 172.21.15.19:0/3299569290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7090100e20 msgr2=0x7f70901984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708effd640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7090100e20 0x7f70901984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708effd640 1 -- 172.21.15.19:0/3299569290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709019d480 con 0x7f7090102570 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708f7fe640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7090100e20 0x7f70901984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708effd640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102570 0x7f7090198a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f708400c920 tx=0x7f708400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708cff9640 1 -- 172.21.15.19:0/3299569290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084007c70 con 0x7f7090102570 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709019d770 con 0x7f7090102570 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708cff9640 1 -- 172.21.15.19:0/3299569290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7084007e10 con 0x7f7090102570 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f708cff9640 1 -- 172.21.15.19:0/3299569290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084018890 con 0x7f7090102570 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.397+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70901a5050 con 0x7f7090102570 2024-09-19T04:04:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.398+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7090101200 con 0x7f7090102570 2024-09-19T04:04:35.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.400+0000 7f708cff9640 1 -- 172.21.15.19:0/3299569290 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7084018a50 con 0x7f7090102570 2024-09-19T04:04:35.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.400+0000 7f708cff9640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70780a3240 0x7f70780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:35.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.400+0000 7f708f7fe640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70780a3240 0x7f70780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:35.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.400+0000 7f708cff9640 1 -- 172.21.15.19:0/3299569290 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f70840cf910 con 0x7f7090102570 2024-09-19T04:04:35.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.401+0000 7f708f7fe640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70780a3240 0x7f70780a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7080009930 tx=0x7f7080008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.403+0000 7f708cff9640 1 -- 172.21.15.19:0/3299569290 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7084014030 con 0x7f7090102570 2024-09-19T04:04:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:35.578+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7090199830 con 0x7f7090102570 2024-09-19T04:04:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.188+0000 7f708cff9640 1 -- 172.21.15.19:0/3299569290 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f708409cb60 con 0x7f7090102570 2024-09-19T04:04:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-09-19T04:04:36.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70780a3240 msgr2=0x7f70780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:36.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70780a3240 0x7f70780a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7080009930 tx=0x7f7080008040 comp rx=0 tx=0).stop 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102570 msgr2=0x7f7090198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102570 0x7f7090198a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f708400c920 tx=0x7f708400cde0 comp rx=0 tx=0).stop 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 shutdown_connections 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70780a3240 0x7f70780a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7090102ef0 0x7f709019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7090102570 0x7f7090198a10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 --2- 172.21.15.19:0/3299569290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7090100e20 0x7f70901984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 >> 172.21.15.19:0/3299569290 conn(0x7f70900fc7d0 msgr2=0x7f709006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 shutdown_connections 2024-09-19T04:04:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:36.190+0000 7f7095da6640 1 -- 172.21.15.19:0/3299569290 wait complete. 2024-09-19T04:04:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:36 smithi196 ceph-mon[35731]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:36 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3299569290' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:04:36.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:36 smithi019 ceph-mon[30938]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:36.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:36 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3299569290' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:04:36.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:36 smithi111 ceph-mon[35365]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:36.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:36 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3299569290' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T04:04:36.900 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-19T04:04:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:37.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:37 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3299569290' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:04:37.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:37 smithi019 ceph-mon[30938]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:04:37.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:37 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3299569290' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:04:37.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:37 smithi196 ceph-mon[35731]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:04:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:37 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3299569290' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T04:04:37.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:37 smithi111 ceph-mon[35365]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T04:04:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 -- 172.21.15.19:0/1936066760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 msgr2=0x7fc3bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/1936066760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 0x7fc3bc10da40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009ca0 tx=0x7fc3b402a950 comp rx=0 tx=0).stop 2024-09-19T04:04:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 -- 172.21.15.19:0/1936066760 shutdown_connections 2024-09-19T04:04:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/1936066760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 0x7fc3bc10da40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/1936066760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3bc102570 0x7fc3bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/1936066760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3bc100e20 0x7fc3bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 -- 172.21.15.19:0/1936066760 >> 172.21.15.19:0/1936066760 conn(0x7fc3bc0fc7d0 msgr2=0x7fc3bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 -- 172.21.15.19:0/1936066760 shutdown_connections 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.667+0000 7fc3c30b3640 1 -- 172.21.15.19:0/1936066760 wait complete. 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c30b3640 1 Processor -- start 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c30b3640 1 -- start start 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c30b3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3bc100e20 0x7fc3bc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c30b3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3bc102570 0x7fc3bc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c30b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 0x7fc3bc1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c30b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc110fa0 con 0x7fc3bc102ef0 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c30b3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc110e20 con 0x7fc3bc102570 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c30b3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc111120 con 0x7fc3bc100e20 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c1650640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 0x7fc3bc1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c1650640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 0x7fc3bc1a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48028/0 (socket says 172.21.15.19:48028) 2024-09-19T04:04:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.668+0000 7fc3c1650640 1 -- 172.21.15.19:0/3632805345 learned_addr learned my addr 172.21.15.19:0/3632805345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3bbfff640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3bc102570 0x7fc3bc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3c0e4f640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3bc100e20 0x7fc3bc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3c1650640 1 -- 172.21.15.19:0/3632805345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3bc100e20 msgr2=0x7fc3bc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3c1650640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3bc100e20 0x7fc3bc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3c1650640 1 -- 172.21.15.19:0/3632805345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3bc102570 msgr2=0x7fc3bc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3c1650640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3bc102570 0x7fc3bc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3c1650640 1 -- 172.21.15.19:0/3632805345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc1a48b0 con 0x7fc3bc102ef0 2024-09-19T04:04:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3c1650640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 0x7fc3bc1a4370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc3b402ae70 tx=0x7fc3b402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3b9ffb640 1 -- 172.21.15.19:0/3632805345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4034040 con 0x7fc3bc102ef0 2024-09-19T04:04:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3b9ffb640 1 -- 172.21.15.19:0/3632805345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b4030920 con 0x7fc3bc102ef0 2024-09-19T04:04:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3bc1a4a80 con 0x7fc3bc102ef0 2024-09-19T04:04:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.669+0000 7fc3b9ffb640 1 -- 172.21.15.19:0/3632805345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4045480 con 0x7fc3bc102ef0 2024-09-19T04:04:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.670+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3bc1a4f30 con 0x7fc3bc102ef0 2024-09-19T04:04:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.670+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3bc101200 con 0x7fc3bc102ef0 2024-09-19T04:04:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.672+0000 7fc3b9ffb640 1 -- 172.21.15.19:0/3632805345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc3b4037020 con 0x7fc3bc102ef0 2024-09-19T04:04:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.672+0000 7fc3b9ffb640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc3ac0a3240 0x7fc3ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.673+0000 7fc3c0e4f640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc3ac0a3240 0x7fc3ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.673+0000 7fc3b9ffb640 1 -- 172.21.15.19:0/3632805345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7497+0+0 (secure 0 0 0) 0x7fc3b40f0690 con 0x7fc3bc102ef0 2024-09-19T04:04:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.673+0000 7fc3c0e4f640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc3ac0a3240 0x7fc3ac0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009890 tx=0x7fc3b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.676+0000 7fc3b9ffb640 1 -- 172.21.15.19:0/3632805345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc3b4081210 con 0x7fc3bc102ef0 2024-09-19T04:04:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:37.853+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fc3bc1a5220 con 0x7fc3bc102ef0 2024-09-19T04:04:38.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.294+0000 7fc3b9ffb640 1 -- 172.21.15.19:0/3632805345 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) ==== 171+0+0 (secure 0 0 0) 0x7fc3b40bd6e0 con 0x7fc3bc102ef0 2024-09-19T04:04:38.296 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-09-19T04:04:38.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc3ac0a3240 msgr2=0x7fc3ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:38.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc3ac0a3240 0x7fc3ac0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009890 tx=0x7fc3b0008040 comp rx=0 tx=0).stop 2024-09-19T04:04:38.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 msgr2=0x7fc3bc1a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:38.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 0x7fc3bc1a4370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc3b402ae70 tx=0x7fc3b402f5d0 comp rx=0 tx=0).stop 2024-09-19T04:04:38.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 shutdown_connections 2024-09-19T04:04:38.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc3ac0a3240 0x7fc3ac0a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:38.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3bc102ef0 0x7fc3bc1a4370 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:38.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3bc102570 0x7fc3bc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:38.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 --2- 172.21.15.19:0/3632805345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3bc100e20 0x7fc3bc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:38.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.296+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 >> 172.21.15.19:0/3632805345 conn(0x7fc3bc0fc7d0 msgr2=0x7fc3bc106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:38.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.297+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 shutdown_connections 2024-09-19T04:04:38.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:38.297+0000 7fc3c30b3640 1 -- 172.21.15.19:0/3632805345 wait complete. 2024-09-19T04:04:38.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:38 smithi019 ceph-mon[30938]: pgmap v277: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:38.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:38 smithi019 ceph-mon[30938]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:04:38.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:38 smithi019 ceph-mon[30938]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:04:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:38 smithi111 ceph-mon[35365]: pgmap v277: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:38 smithi111 ceph-mon[35365]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:04:38.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:38 smithi111 ceph-mon[35365]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:04:38.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:38 smithi196 ceph-mon[35731]: pgmap v277: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:38.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:38 smithi196 ceph-mon[35731]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:04:38.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:38 smithi196 ceph-mon[35731]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T04:04:38.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:04:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:39.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:39 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3632805345' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T04:04:39.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:39 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3632805345' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T04:04:39.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:39 smithi019 ceph-mon[30938]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:04:39.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:39 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3632805345' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T04:04:39.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:39 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3632805345' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T04:04:39.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:39 smithi111 ceph-mon[35365]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:04:39.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:39 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3632805345' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T04:04:39.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:39 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3632805345' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T04:04:39.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:39 smithi196 ceph-mon[35731]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T04:04:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 -- 172.21.15.19:0/769784340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 msgr2=0x7fd068103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 --2- 172.21.15.19:0/769784340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 0x7fd068103180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd064009900 tx=0x7fd06402a480 comp rx=0 tx=0).stop 2024-09-19T04:04:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 -- 172.21.15.19:0/769784340 shutdown_connections 2024-09-19T04:04:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 --2- 172.21.15.19:0/769784340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06810c180 0x7fd06810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 --2- 172.21.15.19:0/769784340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0681036c0 0x7fd06810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 --2- 172.21.15.19:0/769784340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 0x7fd068103180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 -- 172.21.15.19:0/769784340 >> 172.21.15.19:0/769784340 conn(0x7fd0680fc7a0 msgr2=0x7fd0680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 -- 172.21.15.19:0/769784340 shutdown_connections 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 -- 172.21.15.19:0/769784340 wait complete. 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 Processor -- start 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.853+0000 7fd06fab1640 1 -- start start 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06fab1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 0x7fd0681985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06fab1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0681036c0 0x7fd068198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06fab1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06810c180 0x7fd06819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06fab1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd068112020 con 0x7fd068102da0 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06fab1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd068111ea0 con 0x7fd06810c180 2024-09-19T04:04:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06fab1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681121a0 con 0x7fd0681036c0 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 0x7fd0681985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 0x7fd0681985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48034/0 (socket says 172.21.15.19:48034) 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 -- 172.21.15.19:0/1053954235 learned_addr learned my addr 172.21.15.19:0/1053954235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d04c640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0681036c0 0x7fd068198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06e04e640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06810c180 0x7fd06819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 -- 172.21.15.19:0/1053954235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0681036c0 msgr2=0x7fd068198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0681036c0 0x7fd068198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 -- 172.21.15.19:0/1053954235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06810c180 msgr2=0x7fd06819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06810c180 0x7fd06819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 -- 172.21.15.19:0/1053954235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06819d550 con 0x7fd068102da0 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06e04e640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06810c180 0x7fd06819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd06d84d640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 0x7fd0681985a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd064005ab0 tx=0x7fd0640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.854+0000 7fd05effd640 1 -- 172.21.15.19:0/1053954235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06403f030 con 0x7fd068102da0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.855+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06819d7e0 con 0x7fd068102da0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.855+0000 7fd05effd640 1 -- 172.21.15.19:0/1053954235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd064003ce0 con 0x7fd068102da0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.855+0000 7fd05effd640 1 -- 172.21.15.19:0/1053954235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064035070 con 0x7fd068102da0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.855+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0681a5120 con 0x7fd068102da0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.856+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd068103200 con 0x7fd068102da0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.857+0000 7fd05effd640 1 -- 172.21.15.19:0/1053954235 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd064042030 con 0x7fd068102da0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.858+0000 7fd05effd640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd0540a3240 0x7fd0540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.858+0000 7fd06d04c640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd0540a3240 0x7fd0540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.858+0000 7fd05effd640 1 -- 172.21.15.19:0/1053954235 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7879+0+0 (secure 0 0 0) 0x7fd0640f0180 con 0x7fd068102da0 2024-09-19T04:04:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.858+0000 7fd06d04c640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd0540a3240 0x7fd0540a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd0680fee70 tx=0x7fd058009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:39.862+0000 7fd05effd640 1 -- 172.21.15.19:0/1053954235 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0640bcfb0 con 0x7fd068102da0 2024-09-19T04:04:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.076+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd068107200 con 0x7fd068102da0 2024-09-19T04:04:40.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.301+0000 7fd05effd640 1 -- 172.21.15.19:0/1053954235 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd0640bc8b0 con 0x7fd068102da0 2024-09-19T04:04:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T04:04:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd0540a3240 msgr2=0x7fd0540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd0540a3240 0x7fd0540a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd0680fee70 tx=0x7fd058009110 comp rx=0 tx=0).stop 2024-09-19T04:04:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 msgr2=0x7fd0681985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 0x7fd0681985a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd064005ab0 tx=0x7fd0640042f0 comp rx=0 tx=0).stop 2024-09-19T04:04:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 shutdown_connections 2024-09-19T04:04:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd0540a3240 0x7fd0540a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06810c180 0x7fd06819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:40.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0681036c0 0x7fd068198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:40.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 --2- 172.21.15.19:0/1053954235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd068102da0 0x7fd0681985a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:40.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.303+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 >> 172.21.15.19:0/1053954235 conn(0x7fd0680fc7a0 msgr2=0x7fd0680fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:40.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.304+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 shutdown_connections 2024-09-19T04:04:40.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:40.304+0000 7fd06fab1640 1 -- 172.21.15.19:0/1053954235 wait complete. 2024-09-19T04:04:40.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:40 smithi019 ceph-mon[30938]: pgmap v280: 97 pgs: 14 creating+peering, 71 unknown, 12 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:40.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:40 smithi019 ceph-mon[30938]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:04:40.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:40 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1053954235' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:04:40.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:40 smithi019 ceph-mon[30938]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:04:40.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:40 smithi111 ceph-mon[35365]: pgmap v280: 97 pgs: 14 creating+peering, 71 unknown, 12 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:40.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:40 smithi111 ceph-mon[35365]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:04:40.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:40 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1053954235' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:04:40.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:40 smithi111 ceph-mon[35365]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:04:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:40 smithi196 ceph-mon[35731]: pgmap v280: 97 pgs: 14 creating+peering, 71 unknown, 12 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:40 smithi196 ceph-mon[35731]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T04:04:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:40 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1053954235' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T04:04:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:40 smithi196 ceph-mon[35731]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T04:04:41.005 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-19T04:04:41.005 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-19T04:04:41.267 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:41.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:41 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1053954235' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:04:41.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:41 smithi019 ceph-mon[30938]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:04:41.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:41 smithi019 ceph-mon[30938]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:04:41.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:41 smithi019 ceph-mon[30938]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:04:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:41 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1053954235' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:04:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:41 smithi111 ceph-mon[35365]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:04:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:41 smithi111 ceph-mon[35365]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:04:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:41 smithi111 ceph-mon[35365]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:04:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:41 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1053954235' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T04:04:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:41 smithi196 ceph-mon[35731]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T04:04:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:41 smithi196 ceph-mon[35731]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T04:04:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:41 smithi196 ceph-mon[35731]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T04:04:41.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 -- 172.21.15.19:0/2770246143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8102790 msgr2=0x7f58b81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 --2- 172.21.15.19:0/2770246143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8102790 0x7f58b81067e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009900 tx=0x7f58a002a410 comp rx=0 tx=0).stop 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 -- 172.21.15.19:0/2770246143 shutdown_connections 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 --2- 172.21.15.19:0/2770246143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58b8106f10 0x7f58b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 --2- 172.21.15.19:0/2770246143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8102790 0x7f58b81067e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 --2- 172.21.15.19:0/2770246143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b8101de0 0x7f58b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 -- 172.21.15.19:0/2770246143 >> 172.21.15.19:0/2770246143 conn(0x7f58b80fd8c0 msgr2=0x7f58b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 -- 172.21.15.19:0/2770246143 shutdown_connections 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.738+0000 7f58bdacd640 1 -- 172.21.15.19:0/2770246143 wait complete. 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58bdacd640 1 Processor -- start 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58bdacd640 1 -- start start 2024-09-19T04:04:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58bdacd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8101de0 0x7f58b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58bdacd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58b8102790 0x7f58b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58bdacd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b8106f10 0x7f58b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58bdacd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8111f30 con 0x7f58b8101de0 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58bdacd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8111db0 con 0x7f58b8106f10 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58bdacd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58b81120b0 con 0x7f58b8102790 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8101de0 0x7f58b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8101de0 0x7f58b80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33458/0 (socket says 172.21.15.19:33458) 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.739+0000 7f58b77fe640 1 -- 172.21.15.19:0/229744800 learned_addr learned my addr 172.21.15.19:0/229744800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58affff640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58b8102790 0x7f58b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b77fe640 1 -- 172.21.15.19:0/229744800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58b8102790 msgr2=0x7f58b8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b7fff640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b8106f10 0x7f58b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b77fe640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58b8102790 0x7f58b8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b77fe640 1 -- 172.21.15.19:0/229744800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b8106f10 msgr2=0x7f58b8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b77fe640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b8106f10 0x7f58b8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b77fe640 1 -- 172.21.15.19:0/229744800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b81a68f0 con 0x7f58b8101de0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58affff640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58b8102790 0x7f58b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b77fe640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8101de0 0x7f58b80744f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58a800d950 tx=0x7f58a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b57fa640 1 -- 172.21.15.19:0/229744800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8017030 con 0x7f58b8101de0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b57fa640 1 -- 172.21.15.19:0/229744800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a8004020 con 0x7f58b8101de0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b81a6be0 con 0x7f58b8101de0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58b57fa640 1 -- 172.21.15.19:0/229744800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a80047a0 con 0x7f58b8101de0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.740+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b81a7170 con 0x7f58b8101de0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.741+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b81021c0 con 0x7f58b8101de0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.742+0000 7f58b57fa640 1 -- 172.21.15.19:0/229744800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f58a80041d0 con 0x7f58b8101de0 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.743+0000 7f58b57fa640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f58a40a32c0 0x7f58a40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.743+0000 7f58affff640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f58a40a32c0 0x7f58a40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.743+0000 7f58b57fa640 1 -- 172.21.15.19:0/229744800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f58a80cf860 con 0x7f58b8101de0 2024-09-19T04:04:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.743+0000 7f58affff640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f58a40a32c0 0x7f58a40a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000c00 tx=0x7f58a0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.746+0000 7f58b57fa640 1 -- 172.21.15.19:0/229744800 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f58a809c750 con 0x7f58b8101de0 2024-09-19T04:04:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:41.929+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f58b8075a20 con 0x7f58b8101de0 2024-09-19T04:04:42.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.321+0000 7f58b57fa640 1 -- 172.21.15.19:0/229744800 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) ==== 157+0+0 (secure 0 0 0) 0x7f58a809c050 con 0x7f58b8101de0 2024-09-19T04:04:42.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f58a40a32c0 msgr2=0x7f58a40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:42.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f58a40a32c0 0x7f58a40a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000c00 tx=0x7f58a0032c70 comp rx=0 tx=0).stop 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8101de0 msgr2=0x7f58b80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8101de0 0x7f58b80744f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58a800d950 tx=0x7f58a800de10 comp rx=0 tx=0).stop 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 shutdown_connections 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f58a40a32c0 0x7f58a40a5780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b8106f10 0x7f58b8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58b8102790 0x7f58b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 --2- 172.21.15.19:0/229744800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58b8101de0 0x7f58b80744f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 >> 172.21.15.19:0/229744800 conn(0x7f58b80fd8c0 msgr2=0x7f58b80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 shutdown_connections 2024-09-19T04:04:42.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:42.324+0000 7f58bdacd640 1 -- 172.21.15.19:0/229744800 wait complete. 2024-09-19T04:04:42.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:42 smithi111 ceph-mon[35365]: pgmap v283: 97 pgs: 28 creating+peering, 34 unknown, 35 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T04:04:42.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:42 smithi111 ceph-mon[35365]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:04:42.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:42 smithi111 ceph-mon[35365]: Cluster is now healthy 2024-09-19T04:04:42.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:42 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] up:active 2024-09-19T04:04:42.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:42 smithi111 ceph-mon[35365]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:42 smithi111 ceph-mon[35365]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:42 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/229744800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T04:04:42.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:42 smithi019 ceph-mon[30938]: pgmap v283: 97 pgs: 28 creating+peering, 34 unknown, 35 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T04:04:42.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:42 smithi019 ceph-mon[30938]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:04:42.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:42 smithi019 ceph-mon[30938]: Cluster is now healthy 2024-09-19T04:04:42.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:42 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] up:active 2024-09-19T04:04:42.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:42 smithi019 ceph-mon[30938]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:42.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:42 smithi019 ceph-mon[30938]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:42.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:42 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/229744800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T04:04:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:42 smithi196 ceph-mon[35731]: pgmap v283: 97 pgs: 28 creating+peering, 34 unknown, 35 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T04:04:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:42 smithi196 ceph-mon[35731]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T04:04:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:42 smithi196 ceph-mon[35731]: Cluster is now healthy 2024-09-19T04:04:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:42 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] up:active 2024-09-19T04:04:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:42 smithi196 ceph-mon[35731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:42 smithi196 ceph-mon[35731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:42 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/229744800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T04:04:43.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:04:43.307 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:43.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:43 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/229744800' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T04:04:43.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:43 smithi019 ceph-mon[30938]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:04:43.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:43 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/229744800' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T04:04:43.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:43 smithi111 ceph-mon[35365]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:04:43.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:43 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/229744800' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T04:04:43.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:43 smithi196 ceph-mon[35731]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T04:04:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 -- 172.21.15.19:0/1755102913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 msgr2=0x7fdc9010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1755102913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 0x7fdc9010ec30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009cd0 tx=0x7fdc8802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:04:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 -- 172.21.15.19:0/1755102913 shutdown_connections 2024-09-19T04:04:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1755102913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 0x7fdc9010ec30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1755102913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc90103740 0x7fdc90107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1755102913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc90102d90 0x7fdc90103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 -- 172.21.15.19:0/1755102913 >> 172.21.15.19:0/1755102913 conn(0x7fdc900fc7d0 msgr2=0x7fdc900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 -- 172.21.15.19:0/1755102913 shutdown_connections 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 -- 172.21.15.19:0/1755102913 wait complete. 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 Processor -- start 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.807+0000 7fdc973a5640 1 -- start start 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc973a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc90102d90 0x7fdc901986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc973a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc90103740 0x7fdc90198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc973a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 0x7fdc9019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc973a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90112150 con 0x7fdc90108290 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc973a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90111fd0 con 0x7fdc90103740 2024-09-19T04:04:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc973a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901122d0 con 0x7fdc90102d90 2024-09-19T04:04:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 0x7fdc9019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 0x7fdc9019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33484/0 (socket says 172.21.15.19:33484) 2024-09-19T04:04:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 -- 172.21.15.19:0/1564928338 learned_addr learned my addr 172.21.15.19:0/1564928338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc94940640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc90103740 0x7fdc90198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95141640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc90102d90 0x7fdc901986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 -- 172.21.15.19:0/1564928338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc90102d90 msgr2=0x7fdc901986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc90102d90 0x7fdc901986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 -- 172.21.15.19:0/1564928338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc90103740 msgr2=0x7fdc90198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc90103740 0x7fdc90198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 -- 172.21.15.19:0/1564928338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9019d6a0 con 0x7fdc90108290 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.808+0000 7fdc95942640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 0x7fdc9019cfc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006010 tx=0x7fdc88003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.809+0000 7fdc827fc640 1 -- 172.21.15.19:0/1564928338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88004450 con 0x7fdc90108290 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.809+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9019d930 con 0x7fdc90108290 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.809+0000 7fdc827fc640 1 -- 172.21.15.19:0/1564928338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc88003e10 con 0x7fdc90108290 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.809+0000 7fdc827fc640 1 -- 172.21.15.19:0/1564928338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88046480 con 0x7fdc90108290 2024-09-19T04:04:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.809+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc901a5270 con 0x7fdc90108290 2024-09-19T04:04:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.810+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9006a7f0 con 0x7fdc90108290 2024-09-19T04:04:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.811+0000 7fdc827fc640 1 -- 172.21.15.19:0/1564928338 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdc880045f0 con 0x7fdc90108290 2024-09-19T04:04:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.812+0000 7fdc827fc640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc7c0a3240 0x7fdc7c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:43.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.812+0000 7fdc95141640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc7c0a3240 0x7fdc7c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:43.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.812+0000 7fdc827fc640 1 -- 172.21.15.19:0/1564928338 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fdc8802e030 con 0x7fdc90108290 2024-09-19T04:04:43.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.812+0000 7fdc95141640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc7c0a3240 0x7fdc7c0a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009990 tx=0x7fdc8c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:43.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.815+0000 7fdc827fc640 1 -- 172.21.15.19:0/1564928338 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc8804b060 con 0x7fdc90108290 2024-09-19T04:04:43.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:43.994+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdc901031f0 con 0x7fdc90108290 2024-09-19T04:04:45.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:44 smithi111 ceph-mon[35365]: pgmap v285: 97 pgs: 22 creating+peering, 10 unknown, 65 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 249 B/s wr, 6 op/s 2024-09-19T04:04:45.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:44 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1564928338' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:04:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:44 smithi019 ceph-mon[30938]: pgmap v285: 97 pgs: 22 creating+peering, 10 unknown, 65 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 249 B/s wr, 6 op/s 2024-09-19T04:04:45.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:44 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1564928338' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:04:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:44 smithi196 ceph-mon[35731]: pgmap v285: 97 pgs: 22 creating+peering, 10 unknown, 65 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 249 B/s wr, 6 op/s 2024-09-19T04:04:45.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:44 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1564928338' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T04:04:45.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.338+0000 7fdc827fc640 1 -- 172.21.15.19:0/1564928338 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) ==== 228+0+0 (secure 0 0 0) 0x7fdc901031f0 con 0x7fdc90108290 2024-09-19T04:04:45.340 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data_ec' created 2024-09-19T04:04:45.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc7c0a3240 msgr2=0x7fdc7c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc7c0a3240 0x7fdc7c0a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009990 tx=0x7fdc8c009110 comp rx=0 tx=0).stop 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 msgr2=0x7fdc9019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 0x7fdc9019cfc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006010 tx=0x7fdc88003c20 comp rx=0 tx=0).stop 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 shutdown_connections 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc7c0a3240 0x7fdc7c0a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc90108290 0x7fdc9019cfc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc90103740 0x7fdc90198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 --2- 172.21.15.19:0/1564928338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc90102d90 0x7fdc901986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 >> 172.21.15.19:0/1564928338 conn(0x7fdc900fc7d0 msgr2=0x7fdc90105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 shutdown_connections 2024-09-19T04:04:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:45.341+0000 7fdc973a5640 1 -- 172.21.15.19:0/1564928338 wait complete. 2024-09-19T04:04:45.655 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:04:45 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: 2024-09-19T04:04:45.174+0000 7f7034aba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:04:45.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:45 smithi019 ceph-mon[30938]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T04:04:45.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:45 smithi019 ceph-mon[30938]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:04:45.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:45 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1564928338' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:04:45.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:45 smithi019 ceph-mon[30938]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:04:46.006 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-19T04:04:46.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:45 smithi111 ceph-mon[35365]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T04:04:46.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:45 smithi111 ceph-mon[35365]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:04:46.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:45 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1564928338' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:04:46.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:45 smithi111 ceph-mon[35365]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:04:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:45 smithi196 ceph-mon[35731]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T04:04:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:45 smithi196 ceph-mon[35731]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T04:04:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:45 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1564928338' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T04:04:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:45 smithi196 ceph-mon[35731]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T04:04:46.269 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 -- 172.21.15.19:0/705785417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102ef0 msgr2=0x7f43d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 --2- 172.21.15.19:0/705785417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102ef0 0x7f43d010da40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009ca0 tx=0x7f43c802a950 comp rx=0 tx=0).stop 2024-09-19T04:04:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 -- 172.21.15.19:0/705785417 shutdown_connections 2024-09-19T04:04:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 --2- 172.21.15.19:0/705785417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102ef0 0x7f43d010da40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 --2- 172.21.15.19:0/705785417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43d0102570 0x7f43d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 --2- 172.21.15.19:0/705785417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d0100e20 0x7f43d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 -- 172.21.15.19:0/705785417 >> 172.21.15.19:0/705785417 conn(0x7f43d00fc7d0 msgr2=0x7f43d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 -- 172.21.15.19:0/705785417 shutdown_connections 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 -- 172.21.15.19:0/705785417 wait complete. 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.844+0000 7f43d5f62640 1 Processor -- start 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43d5f62640 1 -- start start 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43d5f62640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d0100e20 0x7f43d0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43d5f62640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102570 0x7f43d01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43d5f62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43d0102ef0 0x7f43d019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43d5f62640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0111020 con 0x7f43d0102570 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43d5f62640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0110ea0 con 0x7f43d0102ef0 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43d5f62640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01111a0 con 0x7f43d0100e20 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43ceffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102570 0x7f43d01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43ceffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102570 0x7f43d01977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33508/0 (socket says 172.21.15.19:33508) 2024-09-19T04:04:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43ceffd640 1 -- 172.21.15.19:0/3510736924 learned_addr learned my addr 172.21.15.19:0/3510736924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43cf7fe640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d0100e20 0x7f43d0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43cffff640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43d0102ef0 0x7f43d019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43ceffd640 1 -- 172.21.15.19:0/3510736924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d0100e20 msgr2=0x7f43d0197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43ceffd640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d0100e20 0x7f43d0197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43ceffd640 1 -- 172.21.15.19:0/3510736924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43d0102ef0 msgr2=0x7f43d019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43ceffd640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43d0102ef0 0x7f43d019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43ceffd640 1 -- 172.21.15.19:0/3510736924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d019c7f0 con 0x7f43d0102570 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.845+0000 7f43cffff640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43d0102ef0 0x7f43d019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.846+0000 7f43ceffd640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102570 0x7f43d01977a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43c400b4a0 tx=0x7f43c400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.846+0000 7f43ccff9640 1 -- 172.21.15.19:0/3510736924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4015020 con 0x7f43d0102570 2024-09-19T04:04:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.846+0000 7f43ccff9640 1 -- 172.21.15.19:0/3510736924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c4011d10 con 0x7f43d0102570 2024-09-19T04:04:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.846+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d019cae0 con 0x7f43d0102570 2024-09-19T04:04:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.846+0000 7f43ccff9640 1 -- 172.21.15.19:0/3510736924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c400f2e0 con 0x7f43d0102570 2024-09-19T04:04:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.846+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d019d070 con 0x7f43d0102570 2024-09-19T04:04:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.847+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d01021b0 con 0x7f43d0102570 2024-09-19T04:04:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.848+0000 7f43ccff9640 1 -- 172.21.15.19:0/3510736924 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f43c4018020 con 0x7f43d0102570 2024-09-19T04:04:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.849+0000 7f43ccff9640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f43b80a32c0 0x7f43b80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.849+0000 7f43cf7fe640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f43b80a32c0 0x7f43b80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.849+0000 7f43ccff9640 1 -- 172.21.15.19:0/3510736924 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f43c40cf820 con 0x7f43d0102570 2024-09-19T04:04:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.849+0000 7f43cf7fe640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f43b80a32c0 0x7f43b80a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009990 tx=0x7f43c0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:46.852+0000 7f43ccff9640 1 -- 172.21.15.19:0/3510736924 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43c401f020 con 0x7f43d0102570 2024-09-19T04:04:47.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.027+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f43d0101280 con 0x7f43d0102570 2024-09-19T04:04:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.345+0000 7f43ccff9640 1 -- 172.21.15.19:0/3510736924 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f43d0101280 con 0x7f43d0102570 2024-09-19T04:04:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:set pool 4 allow_ec_overwrites to true 2024-09-19T04:04:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f43b80a32c0 msgr2=0x7f43b80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f43b80a32c0 0x7f43b80a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009990 tx=0x7f43c0009110 comp rx=0 tx=0).stop 2024-09-19T04:04:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102570 msgr2=0x7f43d01977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102570 0x7f43d01977a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43c400b4a0 tx=0x7f43c400b8a0 comp rx=0 tx=0).stop 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 shutdown_connections 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f43b80a32c0 0x7f43b80a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43d0102ef0 0x7f43d019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43d0102570 0x7f43d01977a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 --2- 172.21.15.19:0/3510736924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d0100e20 0x7f43d0197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 >> 172.21.15.19:0/3510736924 conn(0x7f43d00fc7d0 msgr2=0x7f43d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 shutdown_connections 2024-09-19T04:04:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:47.348+0000 7f43d5f62640 1 -- 172.21.15.19:0/3510736924 wait complete. 2024-09-19T04:04:47.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:47 smithi111 ceph-mon[35365]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:04:47.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:47 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3510736924' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T04:04:47.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:47 smithi019 ceph-mon[30938]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:04:47.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:47 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3510736924' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T04:04:47.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:47 smithi196 ceph-mon[35731]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T04:04:47.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:47 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3510736924' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T04:04:47.996 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-19T04:04:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:48.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:48 smithi019 ceph-mon[30938]: pgmap v290: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-09-19T04:04:48.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:48 smithi019 ceph-mon[30938]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:04:48.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:48 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3510736924' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T04:04:48.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:48 smithi019 ceph-mon[30938]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:04:48.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:48 smithi111 ceph-mon[35365]: pgmap v290: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-09-19T04:04:48.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:48 smithi111 ceph-mon[35365]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:04:48.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:48 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3510736924' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T04:04:48.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:48 smithi111 ceph-mon[35365]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:04:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:48 smithi196 ceph-mon[35731]: pgmap v290: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-09-19T04:04:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:48 smithi196 ceph-mon[35731]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T04:04:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:48 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3510736924' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T04:04:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:48 smithi196 ceph-mon[35731]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/1088787507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44100e20 msgr2=0x7f5d44101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/1088787507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44100e20 0x7f5d44101200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009900 tx=0x7f5d4002a480 comp rx=0 tx=0).stop 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/1088787507 shutdown_connections 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/1088787507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d44102ef0 0x7f5d4410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/1088787507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d44102570 0x7f5d441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/1088787507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44100e20 0x7f5d44101200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/1088787507 >> 172.21.15.19:0/1088787507 conn(0x7f5d440fc7d0 msgr2=0x7f5d440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/1088787507 shutdown_connections 2024-09-19T04:04:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/1088787507 wait complete. 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 Processor -- start 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.727+0000 7f5d4b8b4640 1 -- start start 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d4b8b4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d44100e20 0x7f5d4419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d4b8b4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44102570 0x7f5d4419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d4b8b4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d44102ef0 0x7f5d4419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d4b8b4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44110fd0 con 0x7f5d44102570 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d4b8b4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44110e50 con 0x7f5d44102ef0 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d4b8b4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44111150 con 0x7f5d44100e20 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44102570 0x7f5d4419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44102570 0x7f5d4419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33530/0 (socket says 172.21.15.19:33530) 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 -- 172.21.15.19:0/2507699091 learned_addr learned my addr 172.21.15.19:0/2507699091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d49e51640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d44102ef0 0x7f5d4419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d49650640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d44100e20 0x7f5d4419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 -- 172.21.15.19:0/2507699091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d44100e20 msgr2=0x7f5d4419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d44100e20 0x7f5d4419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 -- 172.21.15.19:0/2507699091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d44102ef0 msgr2=0x7f5d4419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d44102ef0 0x7f5d4419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 -- 172.21.15.19:0/2507699091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4419f670 con 0x7f5d44102570 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d49e51640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d44102ef0 0x7f5d4419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d48e4f640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44102570 0x7f5d4419ac00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400c970 tx=0x7f5d3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.728+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/2507699091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34007bb0 con 0x7f5d44102570 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.729+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/2507699091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d34007d50 con 0x7f5d44102570 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.729+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4419f960 con 0x7f5d44102570 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.729+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/2507699091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400f450 con 0x7f5d44102570 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.729+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d441a7240 con 0x7f5d44102570 2024-09-19T04:04:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.730+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d44101200 con 0x7f5d44102570 2024-09-19T04:04:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.731+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/2507699091 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5d34004fe0 con 0x7f5d44102570 2024-09-19T04:04:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.731+0000 7f5d3a7fc640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5d300a3240 0x7f5d300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.732+0000 7f5d49650640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5d300a3240 0x7f5d300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.732+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/2507699091 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f5d340cfe10 con 0x7f5d44102570 2024-09-19T04:04:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.732+0000 7f5d49650640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5d300a3240 0x7f5d300a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d40000c00 tx=0x7f5d40005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.735+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/2507699091 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d34014030 con 0x7f5d44102570 2024-09-19T04:04:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:48.951+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f5d441a7440 con 0x7f5d44102570 2024-09-19T04:04:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.353+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/2507699091 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f5d3409c8b0 con 0x7f5d44102570 2024-09-19T04:04:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:added data pool 4 to fsmap 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.355+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5d300a3240 msgr2=0x7f5d300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.355+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5d300a3240 0x7f5d300a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d40000c00 tx=0x7f5d40005a20 comp rx=0 tx=0).stop 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.355+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44102570 msgr2=0x7f5d4419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.355+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44102570 0x7f5d4419ac00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400c970 tx=0x7f5d3400ce30 comp rx=0 tx=0).stop 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.356+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 shutdown_connections 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.356+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5d300a3240 0x7f5d300a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.356+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d44102ef0 0x7f5d4419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.356+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44102570 0x7f5d4419ac00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.356+0000 7f5d4b8b4640 1 --2- 172.21.15.19:0/2507699091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d44100e20 0x7f5d4419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.356+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 >> 172.21.15.19:0/2507699091 conn(0x7f5d440fc7d0 msgr2=0x7f5d44069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.356+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 shutdown_connections 2024-09-19T04:04:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:49.356+0000 7f5d4b8b4640 1 -- 172.21.15.19:0/2507699091 wait complete. 2024-09-19T04:04:49.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:49 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2507699091' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T04:04:49.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:49 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2507699091' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T04:04:49.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:49 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2507699091' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T04:04:49.980 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:50.243 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:50.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:50 smithi019 ceph-mon[30938]: pgmap v292: 129 pgs: 4 creating+peering, 17 unknown, 108 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 502 B/s wr, 1 op/s 2024-09-19T04:04:50.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:50 smithi019 ceph-mon[30938]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:04:50.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:50 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2507699091' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T04:04:50.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:50 smithi019 ceph-mon[30938]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:50 smithi111 ceph-mon[35365]: pgmap v292: 129 pgs: 4 creating+peering, 17 unknown, 108 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 502 B/s wr, 1 op/s 2024-09-19T04:04:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:50 smithi111 ceph-mon[35365]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:04:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:50 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2507699091' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T04:04:50.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:50 smithi111 ceph-mon[35365]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:50.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:50 smithi196 ceph-mon[35731]: pgmap v292: 129 pgs: 4 creating+peering, 17 unknown, 108 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 502 B/s wr, 1 op/s 2024-09-19T04:04:50.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:50 smithi196 ceph-mon[35731]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T04:04:50.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:50 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2507699091' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T04:04:50.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:50 smithi196 ceph-mon[35731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:04:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/3132552772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4102ef0 msgr2=0x7f1df410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/3132552772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4102ef0 0x7f1df410da40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009ca0 tx=0x7f1dec02a950 comp rx=0 tx=0).stop 2024-09-19T04:04:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/3132552772 shutdown_connections 2024-09-19T04:04:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/3132552772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4102ef0 0x7f1df410da40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/3132552772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1df4102570 0x7f1df41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/3132552772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1df4100e20 0x7f1df4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/3132552772 >> 172.21.15.19:0/3132552772 conn(0x7f1df40fc7d0 msgr2=0x7f1df40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/3132552772 shutdown_connections 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.706+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/3132552772 wait complete. 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.707+0000 7f1dfb1fd640 1 Processor -- start 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.707+0000 7f1dfb1fd640 1 -- start start 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.707+0000 7f1dfb1fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4100e20 0x7f1df4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.707+0000 7f1dfb1fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1df4102570 0x7f1df4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.707+0000 7f1dfb1fd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1df4102ef0 0x7f1df419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1dfb1fd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4110f80 con 0x7f1df4100e20 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1dfb1fd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4110e00 con 0x7f1df4102ef0 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1dfb1fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4111100 con 0x7f1df4102570 2024-09-19T04:04:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4100e20 0x7f1df4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4100e20 0x7f1df4197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33548/0 (socket says 172.21.15.19:33548) 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 -- 172.21.15.19:0/2771901475 learned_addr learned my addr 172.21.15.19:0/2771901475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df3fff640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1df4102570 0x7f1df4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df979a640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1df4102ef0 0x7f1df419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 -- 172.21.15.19:0/2771901475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1df4102570 msgr2=0x7f1df4197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1df4102570 0x7f1df4197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 -- 172.21.15.19:0/2771901475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1df4102ef0 msgr2=0x7f1df419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1df4102ef0 0x7f1df419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 -- 172.21.15.19:0/2771901475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df419c2b0 con 0x7f1df4100e20 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df8f99640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4100e20 0x7f1df4197300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1de800c920 tx=0x7f1de800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df1ffb640 1 -- 172.21.15.19:0/2771901475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8007c70 con 0x7f1df4100e20 2024-09-19T04:04:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1df1ffb640 1 -- 172.21.15.19:0/2771901475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de8007e10 con 0x7f1df4100e20 2024-09-19T04:04:50.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.708+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df419c5a0 con 0x7f1df4100e20 2024-09-19T04:04:50.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.709+0000 7f1df1ffb640 1 -- 172.21.15.19:0/2771901475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8018890 con 0x7f1df4100e20 2024-09-19T04:04:50.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.709+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df4069f70 con 0x7f1df4100e20 2024-09-19T04:04:50.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.710+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df4101200 con 0x7f1df4100e20 2024-09-19T04:04:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.711+0000 7f1df1ffb640 1 -- 172.21.15.19:0/2771901475 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1de8018b30 con 0x7f1df4100e20 2024-09-19T04:04:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.711+0000 7f1df1ffb640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1de40a32c0 0x7f1de40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.711+0000 7f1df3fff640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1de40a32c0 0x7f1de40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.711+0000 7f1df1ffb640 1 -- 172.21.15.19:0/2771901475 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1de80cfb90 con 0x7f1df4100e20 2024-09-19T04:04:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.712+0000 7f1df3fff640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1de40a32c0 0x7f1de40a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1df40fec10 tx=0x7f1de000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.715+0000 7f1df1ffb640 1 -- 172.21.15.19:0/2771901475 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1de8014030 con 0x7f1df4100e20 2024-09-19T04:04:50.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.929+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1df406a230 con 0x7f1df4100e20 2024-09-19T04:04:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.930+0000 7f1df1ffb640 1 -- 172.21.15.19:0/2771901475 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11845 (secure 0 0 0) 0x7f1de8018e20 con 0x7f1df4100e20 2024-09-19T04:04:50.932 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:50.932 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"btime":"2024-09-19T04:04:49:347557+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:04:48.953291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24476,"qdb_cluster":[24476]},"id":1}]} 2024-09-19T04:04:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1de40a32c0 msgr2=0x7f1de40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1de40a32c0 0x7f1de40a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1df40fec10 tx=0x7f1de000b390 comp rx=0 tx=0).stop 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4100e20 msgr2=0x7f1df4197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4100e20 0x7f1df4197300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1de800c920 tx=0x7f1de800cde0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 shutdown_connections 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1de40a32c0 0x7f1de40a5780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1df4102ef0 0x7f1df419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1df4102570 0x7f1df4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 --2- 172.21.15.19:0/2771901475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1df4100e20 0x7f1df4197300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 >> 172.21.15.19:0/2771901475 conn(0x7f1df40fc7d0 msgr2=0x7f1df41067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 shutdown_connections 2024-09-19T04:04:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:50.932+0000 7f1dfb1fd640 1 -- 172.21.15.19:0/2771901475 wait complete. 2024-09-19T04:04:51.555 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:51 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2771901475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:51.583 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:04:51.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:51 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2771901475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:51.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:51 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2771901475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:51.841 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 -- 172.21.15.19:0/824446972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638101420 msgr2=0x7fc63810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/824446972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638101420 0x7fc63810d950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc630009cd0 tx=0x7fc63002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 -- 172.21.15.19:0/824446972 shutdown_connections 2024-09-19T04:04:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/824446972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638101420 0x7fc63810d950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/824446972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638100a80 0x7fc638100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/824446972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc638106a80 0x7fc638106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 -- 172.21.15.19:0/824446972 >> 172.21.15.19:0/824446972 conn(0x7fc6380fc7a0 msgr2=0x7fc6380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 -- 172.21.15.19:0/824446972 shutdown_connections 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.355+0000 7fc63e7e1640 1 -- 172.21.15.19:0/824446972 wait complete. 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc63e7e1640 1 Processor -- start 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc63e7e1640 1 -- start start 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc63e7e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638100a80 0x7fc6380fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc63e7e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638101420 0x7fc6380fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc63e7e1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc638106a80 0x7fc6380fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc63e7e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc638110e80 con 0x7fc638100a80 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc63e7e1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc638110d00 con 0x7fc638106a80 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc63e7e1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc638111000 con 0x7fc638101420 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc637fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638100a80 0x7fc6380fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc637fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638100a80 0x7fc6380fcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59682/0 (socket says 172.21.15.19:59682) 2024-09-19T04:04:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.356+0000 7fc637fff640 1 -- 172.21.15.19:0/713619625 learned_addr learned my addr 172.21.15.19:0/713619625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc63cd7e640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc638106a80 0x7fc6380fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc62ffff640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638101420 0x7fc6380fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc637fff640 1 -- 172.21.15.19:0/713619625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638101420 msgr2=0x7fc6380fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc637fff640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638101420 0x7fc6380fd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc637fff640 1 -- 172.21.15.19:0/713619625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc638106a80 msgr2=0x7fc6380fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc637fff640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc638106a80 0x7fc6380fd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc637fff640 1 -- 172.21.15.19:0/713619625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6381a6960 con 0x7fc638100a80 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc637fff640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638100a80 0x7fc6380fcd60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc62800d950 tx=0x7fc62800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc635ffb640 1 -- 172.21.15.19:0/713619625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628017030 con 0x7fc638100a80 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc635ffb640 1 -- 172.21.15.19:0/713619625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc628004020 con 0x7fc638100a80 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc635ffb640 1 -- 172.21.15.19:0/713619625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6280047a0 con 0x7fc638100a80 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6381a6c50 con 0x7fc638100a80 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.357+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6381a7140 con 0x7fc638100a80 2024-09-19T04:04:52.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.358+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc638100ee0 con 0x7fc638100a80 2024-09-19T04:04:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.359+0000 7fc635ffb640 1 -- 172.21.15.19:0/713619625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc6280041d0 con 0x7fc638100a80 2024-09-19T04:04:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.360+0000 7fc635ffb640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6240a32c0 0x7fc6240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.360+0000 7fc62ffff640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6240a32c0 0x7fc6240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.360+0000 7fc635ffb640 1 -- 172.21.15.19:0/713619625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc6280cfae0 con 0x7fc638100a80 2024-09-19T04:04:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.360+0000 7fc62ffff640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6240a32c0 0x7fc6240a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc620005fd0 tx=0x7fc620005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.363+0000 7fc635ffb640 1 -- 172.21.15.19:0/713619625 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc62809c640 con 0x7fc638100a80 2024-09-19T04:04:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.540+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6380fe290 con 0x7fc638100a80 2024-09-19T04:04:52.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:52 smithi019 ceph-mon[30938]: pgmap v294: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:52.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:52 smithi019 ceph-mon[30938]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T04:04:52.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:52 smithi019 ceph-mon[30938]: Cluster is now healthy 2024-09-19T04:04:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.542+0000 7fc635ffb640 1 -- 172.21.15.19:0/713619625 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24188 (secure 0 0 0) 0x7fc62809bf40 con 0x7fc638100a80 2024-09-19T04:04:52.543 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:52.544 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:04:48.953123+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:04:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6240a32c0 msgr2=0x7fc6240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6240a32c0 0x7fc6240a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc620005fd0 tx=0x7fc620005e20 comp rx=0 tx=0).stop 2024-09-19T04:04:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638100a80 msgr2=0x7fc6380fcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638100a80 0x7fc6380fcd60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc62800d950 tx=0x7fc62800de10 comp rx=0 tx=0).stop 2024-09-19T04:04:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 shutdown_connections 2024-09-19T04:04:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6240a32c0 0x7fc6240a5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc638106a80 0x7fc6380fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638101420 0x7fc6380fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 --2- 172.21.15.19:0/713619625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc638100a80 0x7fc6380fcd60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 >> 172.21.15.19:0/713619625 conn(0x7fc6380fc7a0 msgr2=0x7fc63806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 shutdown_connections 2024-09-19T04:04:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:52.544+0000 7fc63e7e1640 1 -- 172.21.15.19:0/713619625 wait complete. 2024-09-19T04:04:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:52 smithi111 ceph-mon[35365]: pgmap v294: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:52 smithi111 ceph-mon[35365]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T04:04:52.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:52 smithi111 ceph-mon[35365]: Cluster is now healthy 2024-09-19T04:04:52.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:52 smithi196 ceph-mon[35731]: pgmap v294: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:52.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:52 smithi196 ceph-mon[35731]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T04:04:52.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:52 smithi196 ceph-mon[35731]: Cluster is now healthy 2024-09-19T04:04:53.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:04:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:53.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:53 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/713619625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:53.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:53 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/713619625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:53.654 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:53 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/713619625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 -- 172.21.15.19:0/2935538546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 msgr2=0x7f34bc100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 --2- 172.21.15.19:0/2935538546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 0x7f34bc100730 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009930 tx=0x7f34ac02a420 comp rx=0 tx=0).stop 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 -- 172.21.15.19:0/2935538546 shutdown_connections 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 --2- 172.21.15.19:0/2935538546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34bc106f50 0x7f34bc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 --2- 172.21.15.19:0/2935538546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc100d00 0x7f34bc106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 --2- 172.21.15.19:0/2935538546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 0x7f34bc100730 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 -- 172.21.15.19:0/2935538546 >> 172.21.15.19:0/2935538546 conn(0x7f34bc077210 msgr2=0x7f34bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 -- 172.21.15.19:0/2935538546 shutdown_connections 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.888+0000 7f34c2243640 1 -- 172.21.15.19:0/2935538546 wait complete. 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34c2243640 1 Processor -- start 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34c2243640 1 -- start start 2024-09-19T04:04:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34c2243640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 0x7f34bc198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34c2243640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc100d00 0x7f34bc198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34c2243640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34bc106f50 0x7f34bc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34c2243640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc112080 con 0x7f34bc100350 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34c2243640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc111f00 con 0x7f34bc106f50 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34c2243640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc112200 con 0x7f34bc100d00 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 0x7f34bc198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 0x7f34bc198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59696/0 (socket says 172.21.15.19:59696) 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34bb7fe640 1 -- 172.21.15.19:0/2861817110 learned_addr learned my addr 172.21.15.19:0/2861817110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.889+0000 7f34baffd640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc100d00 0x7f34bc198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34bbfff640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34bc106f50 0x7f34bc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34bb7fe640 1 -- 172.21.15.19:0/2861817110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc100d00 msgr2=0x7f34bc198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34bb7fe640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc100d00 0x7f34bc198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34bb7fe640 1 -- 172.21.15.19:0/2861817110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34bc106f50 msgr2=0x7f34bc19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34bb7fe640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34bc106f50 0x7f34bc19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34bb7fe640 1 -- 172.21.15.19:0/2861817110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc19d5f0 con 0x7f34bc100350 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34bb7fe640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 0x7f34bc198640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009a50 tx=0x7f34ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34b8ff9640 1 -- 172.21.15.19:0/2861817110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac03f030 con 0x7f34bc100350 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34b8ff9640 1 -- 172.21.15.19:0/2861817110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ac003ad0 con 0x7f34bc100350 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc19d880 con 0x7f34bc100350 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34b8ff9640 1 -- 172.21.15.19:0/2861817110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac035070 con 0x7f34bc100350 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.890+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1a51c0 con 0x7f34bc100350 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.891+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc103410 con 0x7f34bc100350 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.892+0000 7f34b8ff9640 1 -- 172.21.15.19:0/2861817110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f34ac042030 con 0x7f34bc100350 2024-09-19T04:04:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.893+0000 7f34b8ff9640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34a40a3240 0x7f34a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.893+0000 7f34b8ff9640 1 -- 172.21.15.19:0/2861817110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f34ac0f0330 con 0x7f34bc100350 2024-09-19T04:04:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.893+0000 7f34baffd640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34a40a3240 0x7f34a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.893+0000 7f34baffd640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34a40a3240 0x7f34a40a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34bc1049a0 tx=0x7f34b0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:53.897+0000 7f34b8ff9640 1 -- 172.21.15.19:0/2861817110 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34ac0bce10 con 0x7f34bc100350 2024-09-19T04:04:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.072+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34bc10a120 con 0x7f34bc100350 2024-09-19T04:04:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.075+0000 7f34b8ff9640 1 -- 172.21.15.19:0/2861817110 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24188 (secure 0 0 0) 0x7f34ac0bc710 con 0x7f34bc100350 2024-09-19T04:04:54.076 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:54.077 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:04:48.953123+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:04:54.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34a40a3240 msgr2=0x7f34a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34a40a3240 0x7f34a40a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34bc1049a0 tx=0x7f34b0009110 comp rx=0 tx=0).stop 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 msgr2=0x7f34bc198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 0x7f34bc198640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009a50 tx=0x7f34ac004270 comp rx=0 tx=0).stop 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 shutdown_connections 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34a40a3240 0x7f34a40a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34bc106f50 0x7f34bc19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc100d00 0x7f34bc198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 --2- 172.21.15.19:0/2861817110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34bc100350 0x7f34bc198640 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 >> 172.21.15.19:0/2861817110 conn(0x7f34bc077210 msgr2=0x7f34bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 shutdown_connections 2024-09-19T04:04:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:54.077+0000 7f34c2243640 1 -- 172.21.15.19:0/2861817110 wait complete. 2024-09-19T04:04:54.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:54 smithi111 ceph-mon[35365]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:54.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:54 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2861817110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:54.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:54 smithi019 ceph-mon[30938]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:54.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:54 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2861817110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:54.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:54 smithi196 ceph-mon[35731]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:54.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:54 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2861817110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:54.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:04:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:55.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 -- 172.21.15.19:0/135510739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0102790 msgr2=0x7f5ca01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 --2- 172.21.15.19:0/135510739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0102790 0x7f5ca01067e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009900 tx=0x7f5c9002a410 comp rx=0 tx=0).stop 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 -- 172.21.15.19:0/135510739 shutdown_connections 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 --2- 172.21.15.19:0/135510739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca0106f10 0x7f5ca010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 --2- 172.21.15.19:0/135510739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0102790 0x7f5ca01067e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 --2- 172.21.15.19:0/135510739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca0101de0 0x7f5ca01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 -- 172.21.15.19:0/135510739 >> 172.21.15.19:0/135510739 conn(0x7f5ca00fd8c0 msgr2=0x7f5ca00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 -- 172.21.15.19:0/135510739 shutdown_connections 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.595+0000 7f5ca864e640 1 -- 172.21.15.19:0/135510739 wait complete. 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca864e640 1 Processor -- start 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca864e640 1 -- start start 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca864e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0101de0 0x7f5ca019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca864e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca0102790 0x7f5ca019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca864e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca0106f10 0x7f5ca019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca864e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0111f70 con 0x7f5ca0101de0 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca864e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0111df0 con 0x7f5ca0102790 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca864e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01120f0 con 0x7f5ca0106f10 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca63ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0101de0 0x7f5ca019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca63ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0101de0 0x7f5ca019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59714/0 (socket says 172.21.15.19:59714) 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.596+0000 7f5ca63ea640 1 -- 172.21.15.19:0/2344561256 learned_addr learned my addr 172.21.15.19:0/2344561256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca5be9640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca0102790 0x7f5ca019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca6beb640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca0106f10 0x7f5ca019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca63ea640 1 -- 172.21.15.19:0/2344561256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca0106f10 msgr2=0x7f5ca019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca63ea640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca0106f10 0x7f5ca019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca63ea640 1 -- 172.21.15.19:0/2344561256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca0102790 msgr2=0x7f5ca019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca63ea640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca0102790 0x7f5ca019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca63ea640 1 -- 172.21.15.19:0/2344561256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca019f5f0 con 0x7f5ca0101de0 2024-09-19T04:04:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca5be9640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca0102790 0x7f5ca019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca63ea640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0101de0 0x7f5ca019a640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00e9f0 tx=0x7f5c9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5c977fe640 1 -- 172.21.15.19:0/2344561256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00cd80 con 0x7f5ca0101de0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5c977fe640 1 -- 172.21.15.19:0/2344561256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c010910 con 0x7f5ca0101de0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca019f8e0 con 0x7f5ca0101de0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5c977fe640 1 -- 172.21.15.19:0/2344561256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c018990 con 0x7f5ca0101de0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.597+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca01a71c0 con 0x7f5ca0101de0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.598+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca01021c0 con 0x7f5ca0101de0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.599+0000 7f5c977fe640 1 -- 172.21.15.19:0/2344561256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5c9c010ac0 con 0x7f5ca0101de0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.600+0000 7f5c977fe640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5c8c0a32c0 0x7f5c8c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.600+0000 7f5ca5be9640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5c8c0a32c0 0x7f5c8c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.600+0000 7f5c977fe640 1 -- 172.21.15.19:0/2344561256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5c9c0d0340 con 0x7f5ca0101de0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.601+0000 7f5ca5be9640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5c8c0a32c0 0x7f5c8c0a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00fe130 tx=0x7f5c90005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.603+0000 7f5c977fe640 1 -- 172.21.15.19:0/2344561256 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c9c09ce20 con 0x7f5ca0101de0 2024-09-19T04:04:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.816+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ca01a73f0 con 0x7f5ca0101de0 2024-09-19T04:04:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.816+0000 7f5c977fe640 1 -- 172.21.15.19:0/2344561256 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11845 (secure 0 0 0) 0x7f5c9c09c720 con 0x7f5ca0101de0 2024-09-19T04:04:55.819 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:55.819 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"btime":"2024-09-19T04:04:49:347557+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:04:48.953291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24476,"qdb_cluster":[24476]},"id":1}]} 2024-09-19T04:04:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5c8c0a32c0 msgr2=0x7f5c8c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5c8c0a32c0 0x7f5c8c0a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00fe130 tx=0x7f5c90005a20 comp rx=0 tx=0).stop 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0101de0 msgr2=0x7f5ca019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0101de0 0x7f5ca019a640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00e9f0 tx=0x7f5c9c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 shutdown_connections 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5c8c0a32c0 0x7f5c8c0a5780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca0106f10 0x7f5ca019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca0102790 0x7f5ca019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 --2- 172.21.15.19:0/2344561256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca0101de0 0x7f5ca019a640 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 >> 172.21.15.19:0/2344561256 conn(0x7f5ca00fd8c0 msgr2=0x7f5ca00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 shutdown_connections 2024-09-19T04:04:55.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:55.819+0000 7f5ca864e640 1 -- 172.21.15.19:0/2344561256 wait complete. 2024-09-19T04:04:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:56 smithi019 ceph-mon[30938]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:56 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2344561256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:56.602 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:04:56.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:56 smithi111 ceph-mon[35365]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:56.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:56 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2344561256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:56.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:56 smithi196 ceph-mon[35731]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:56.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:56 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2344561256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:04:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/1522607729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc1036c0 msgr2=0x7f1dcc10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/1522607729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc1036c0 0x7f1dcc10bbb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009900 tx=0x7f1dbc02a480 comp rx=0 tx=0).stop 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/1522607729 shutdown_connections 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/1522607729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dcc10c180 0x7f1dcc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/1522607729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc1036c0 0x7f1dcc10bbb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/1522607729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dcc102da0 0x7f1dcc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/1522607729 >> 172.21.15.19:0/1522607729 conn(0x7f1dcc0fc7a0 msgr2=0x7f1dcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/1522607729 shutdown_connections 2024-09-19T04:04:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.361+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/1522607729 wait complete. 2024-09-19T04:04:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.362+0000 7f1dd3bbf640 1 Processor -- start 2024-09-19T04:04:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.362+0000 7f1dd3bbf640 1 -- start start 2024-09-19T04:04:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.362+0000 7f1dd3bbf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dcc102da0 0x7f1dcc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.362+0000 7f1dd3bbf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dcc1036c0 0x7f1dcc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.362+0000 7f1dd3bbf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc10c180 0x7f1dcc19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd3bbf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc111ff0 con 0x7f1dcc10c180 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd3bbf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc111e70 con 0x7f1dcc1036c0 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd3bbf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc112170 con 0x7f1dcc102da0 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc10c180 0x7f1dcc19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd115a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dcc1036c0 0x7f1dcc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc10c180 0x7f1dcc19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59734/0 (socket says 172.21.15.19:59734) 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd115a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dcc1036c0 0x7f1dcc1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:48324/0 (socket says 172.21.15.19:48324) 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 -- 172.21.15.19:0/148645237 learned_addr learned my addr 172.21.15.19:0/148645237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd195b640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dcc102da0 0x7f1dcc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 -- 172.21.15.19:0/148645237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dcc102da0 msgr2=0x7f1dcc1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dcc102da0 0x7f1dcc1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 -- 172.21.15.19:0/148645237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dcc1036c0 msgr2=0x7f1dcc1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dcc1036c0 0x7f1dcc1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 -- 172.21.15.19:0/148645237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dcc19d450 con 0x7f1dcc10c180 2024-09-19T04:04:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd195b640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dcc102da0 0x7f1dcc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd115a640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dcc1036c0 0x7f1dcc1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:04:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dd215c640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc10c180 0x7f1dcc19cd70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400eaa0 tx=0x7f1dc400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.363+0000 7f1dc2ffd640 1 -- 172.21.15.19:0/148645237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400cdb0 con 0x7f1dcc10c180 2024-09-19T04:04:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.364+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dcc19d740 con 0x7f1dcc10c180 2024-09-19T04:04:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.364+0000 7f1dc2ffd640 1 -- 172.21.15.19:0/148645237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc4005020 con 0x7f1dcc10c180 2024-09-19T04:04:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.364+0000 7f1dc2ffd640 1 -- 172.21.15.19:0/148645237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40116b0 con 0x7f1dcc10c180 2024-09-19T04:04:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.364+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dcc1a5020 con 0x7f1dcc10c180 2024-09-19T04:04:57.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.364+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dcc103180 con 0x7f1dcc10c180 2024-09-19T04:04:57.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.366+0000 7f1dc2ffd640 1 -- 172.21.15.19:0/148645237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1dc4005560 con 0x7f1dcc10c180 2024-09-19T04:04:57.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.366+0000 7f1dc2ffd640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1db80a3240 0x7f1db80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:57.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.366+0000 7f1dd195b640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1db80a3240 0x7f1db80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:57.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.366+0000 7f1dc2ffd640 1 -- 172.21.15.19:0/148645237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1dc40cfdd0 con 0x7f1dcc10c180 2024-09-19T04:04:57.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.367+0000 7f1dd195b640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1db80a3240 0x7f1db80a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009930 tx=0x7f1dc8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.371+0000 7f1dc2ffd640 1 -- 172.21.15.19:0/148645237 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1dc4016030 con 0x7f1dcc10c180 2024-09-19T04:04:57.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.574+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dcc107220 con 0x7f1dcc10c180 2024-09-19T04:04:57.576 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:57 smithi019 ceph-mon[30938]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:57.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.575+0000 7f1dc2ffd640 1 -- 172.21.15.19:0/148645237 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24188 (secure 0 0 0) 0x7f1dc409c8b0 con 0x7f1dcc10c180 2024-09-19T04:04:57.578 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:04:57.578 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:04:48.953123+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:04:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1db80a3240 msgr2=0x7f1db80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1db80a3240 0x7f1db80a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009930 tx=0x7f1dc8008040 comp rx=0 tx=0).stop 2024-09-19T04:04:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc10c180 msgr2=0x7f1dcc19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc10c180 0x7f1dcc19cd70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400eaa0 tx=0x7f1dc400ef60 comp rx=0 tx=0).stop 2024-09-19T04:04:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 shutdown_connections 2024-09-19T04:04:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1db80a3240 0x7f1db80a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dcc10c180 0x7f1dcc19cd70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dcc1036c0 0x7f1dcc1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 --2- 172.21.15.19:0/148645237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dcc102da0 0x7f1dcc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 >> 172.21.15.19:0/148645237 conn(0x7f1dcc0fc7a0 msgr2=0x7f1dcc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 shutdown_connections 2024-09-19T04:04:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:57.578+0000 7f1dd3bbf640 1 -- 172.21.15.19:0/148645237 wait complete. 2024-09-19T04:04:57.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:57 smithi196 ceph-mon[35731]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:57.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:57 smithi111 ceph-mon[35365]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:58.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:04:40.077653+0000', 'modified': '2024-09-19T04:04:48.953291+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.111:6839/528342006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 528342006}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 528342006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24476, 'qdb_cluster': [24476]} 2024-09-19T04:04:58.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:04:58.253 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:04:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:04:58.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:58 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/148645237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:58.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:58 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/148645237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:58.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:58 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/148645237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:04:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.979+0000 7fc2a4e76640 1 -- 172.21.15.19:0/3228321456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102ef0 msgr2=0x7fc2a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.979+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/3228321456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102ef0 0x7fc2a010da40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc298009ca0 tx=0x7fc29802a950 comp rx=0 tx=0).stop 2024-09-19T04:04:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.979+0000 7fc2a4e76640 1 -- 172.21.15.19:0/3228321456 shutdown_connections 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/3228321456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102ef0 0x7fc2a010da40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/3228321456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a0102570 0x7fc2a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/3228321456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2a0100e20 0x7fc2a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 -- 172.21.15.19:0/3228321456 >> 172.21.15.19:0/3228321456 conn(0x7fc2a00fc7d0 msgr2=0x7fc2a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 -- 172.21.15.19:0/3228321456 shutdown_connections 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 -- 172.21.15.19:0/3228321456 wait complete. 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 Processor -- start 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 -- start start 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.980+0000 7fc2a4e76640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2a0100e20 0x7fc2a0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2a4e76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102570 0x7fc2a0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2a4e76640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a0102ef0 0x7fc2a019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2a4e76640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0111020 con 0x7fc2a0102570 2024-09-19T04:04:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2a4e76640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0110ea0 con 0x7fc2a0102ef0 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2a4e76640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a01111a0 con 0x7fc2a0100e20 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102570 0x7fc2a0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102570 0x7fc2a0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59754/0 (socket says 172.21.15.19:59754) 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 -- 172.21.15.19:0/1851519919 learned_addr learned my addr 172.21.15.19:0/1851519919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc29ed9d640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a0102ef0 0x7fc2a019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc29e59c640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2a0100e20 0x7fc2a0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 -- 172.21.15.19:0/1851519919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2a0100e20 msgr2=0x7fc2a0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2a0100e20 0x7fc2a0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 -- 172.21.15.19:0/1851519919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a0102ef0 msgr2=0x7fc2a019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a0102ef0 0x7fc2a019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 -- 172.21.15.19:0/1851519919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a019d620 con 0x7fc2a0102570 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc29ed9d640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a0102ef0 0x7fc2a019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc29e59c640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2a0100e20 0x7fc2a0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc297fff640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102570 0x7fc2a0198bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc28800b4a0 tx=0x7fc28800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2977fe640 1 -- 172.21.15.19:0/1851519919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288014070 con 0x7fc2a0102570 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2977fe640 1 -- 172.21.15.19:0/1851519919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc288010af0 con 0x7fc2a0102570 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2a4e76640 1 -- 172.21.15.19:0/1851519919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a019d910 con 0x7fc2a0102570 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2977fe640 1 -- 172.21.15.19:0/1851519919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288017d10 con 0x7fc2a0102570 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.981+0000 7fc2a4e76640 1 -- 172.21.15.19:0/1851519919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a01a51f0 con 0x7fc2a0102570 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2977fe640 1 -- 172.21.15.19:0/1851519919 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc288017070 con 0x7fc2a0102570 2024-09-19T04:04:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2a724f740 1 Processor -- start 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2a724f740 1 -- start start 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x556412046db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 0x556412051670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556412051bb0 0x556412053fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2a724f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556411fad090 con 0x556412051bb0 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2a724f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556411f54a70 con 0x5564120469d0 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc2a724f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556411fa9050 con 0x5564120472f0 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc297fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556412051bb0 0x556412053fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc297fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556412051bb0 0x556412053fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59768/0 (socket says 172.21.15.19:59768) 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.984+0000 7fc297fff640 1 -- 172.21.15.19:0/1177261563 learned_addr learned my addr 172.21.15.19:0/1177261563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc29ed9d640 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x556412046db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc29e59c640 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 0x556412051670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc2977fe640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc28c0a32c0 0x7fc28c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc29ed9d640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc28c0a32c0 0x7fc28c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc297fff640 1 -- 172.21.15.19:0/1177261563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 msgr2=0x556412051670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc297fff640 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 0x556412051670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc297fff640 1 -- 172.21.15.19:0/1177261563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 msgr2=0x556412046db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc297fff640 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x556412046db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc297fff640 1 -- 172.21.15.19:0/1177261563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556411f61130 con 0x556412051bb0 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc2977fe640 1 -- 172.21.15.19:0/1851519919 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc2880ceef0 con 0x7fc2a0102570 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc297fff640 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556412051bb0 0x556412053fa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc288059a60 tx=0x7fc288020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5781b37daa58bcc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc29ed9d640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc28c0a32c0 0x7fc28c0a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc298005fc0 tx=0x7fc29802a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc294ff9640 1 -- 172.21.15.19:0/1177261563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28809ae10 con 0x556412051bb0 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc294ff9640 1 -- 172.21.15.19:0/1177261563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc28809b9d0 con 0x556412051bb0 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.985+0000 7fc294ff9640 1 -- 172.21.15.19:0/1177261563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288013170 con 0x556412051bb0 2024-09-19T04:04:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 -- 172.21.15.19:0/1177261563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556412051bb0 msgr2=0x556412053fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556412051bb0 0x556412053fa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc288059a60 tx=0x7fc288020000 comp rx=0 tx=0).stop 2024-09-19T04:04:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 -- 172.21.15.19:0/1177261563 shutdown_connections 2024-09-19T04:04:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556412051bb0 0x556412053fa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 0x556412051670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1177261563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x556412046db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 -- 172.21.15.19:0/1177261563 >> 172.21.15.19:0/1177261563 conn(0x556412042f60 msgr2=0x556412043340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 -- 172.21.15.19:0/1177261563 shutdown_connections 2024-09-19T04:04:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.986+0000 7fc2a724f740 1 -- 172.21.15.19:0/1177261563 wait complete. 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 1 -- start start 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 1 Processor -- start 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120469d0 0x556412051400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 0x55641204e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556412051bb0 0x55641204ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556412054780 con 0x5564120469d0 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556411f67f20 con 0x556412051bb0 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556412054900 con 0x5564120472f0 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc2a724f740 10 client.0 _ll_register_callbacks cb 0x556411f31030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc29e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120469d0 0x556412051400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc29e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120469d0 0x556412051400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59770/0 (socket says 172.21.15.19:59770) 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc29e59c640 1 -- 172.21.15.19:0/133981914 learned_addr learned my addr 172.21.15.19:0/133981914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc297fff640 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 0x55641204e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc29e59c640 1 -- 172.21.15.19:0/133981914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 msgr2=0x55641204e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc29e59c640 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 0x55641204e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc29e59c640 1 -- 172.21.15.19:0/133981914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556412051bb0 msgr2=0x55641204ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc29e59c640 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556412051bb0 0x55641204ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.987+0000 7fc29e59c640 1 -- 172.21.15.19:0/133981914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556411d1e2b0 con 0x5564120469d0 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc29e59c640 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120469d0 0x556412051400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc29000f4f0 tx=0x7fc29000f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ad895dd1758988 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc285ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc285ffb640 1 -- 172.21.15.19:0/133981914 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290013850 con 0x5564120469d0 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc285ffb640 1 -- 172.21.15.19:0/133981914 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc2900139f0 con 0x5564120469d0 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 1 -- 172.21.15.19:0/133981914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc28809b9d0 con 0x5564120469d0 2024-09-19T04:04:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc285ffb640 1 -- 172.21.15.19:0/133981914 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc290013bf0 con 0x5564120469d0 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 10 client.14685 fetch_fsmap learned FSMap version 17 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 1 -- 172.21.15.19:0/133981914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5564120600b0 con 0x5564120469d0 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc285ffb640 1 -- 172.21.15.19:0/133981914 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc28809b9d0 con 0x5564120469d0 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc285ffb640 1 -- 172.21.15.19:0/133981914 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fc2900113e0 con 0x5564120469d0 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 10 client.14685 fetch_fsmap finished waiting for FSMap version 17 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 10 client.14685 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 20 client.14685 populate_metadata read hostname 'smithi019' 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556412062730 0x556412063ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 4 client.14685 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.988+0000 7fc2a724f740 1 -- 172.21.15.19:0/133981914 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556411fba600 con 0x556412062730 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.989+0000 7fc297fff640 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556412062730 0x556412063ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.989+0000 7fc285ffb640 10 client.14685 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:04:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.989+0000 7fc297fff640 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556412062730 0x556412063ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a429ffb471fffd42 server_cookie=d2d96f89b6f144a9 in_seq=0 out_seq=0 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc285ffb640 1 -- 172.21.15.19:0/133981914 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556411fba600 con 0x556412062730 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc285ffb640 10 client.14685 handle_command_reply: tid=0 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 client.14685 shutdown 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 -- 172.21.15.19:0/133981914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120469d0 msgr2=0x556412051400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120469d0 0x556412051400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc29000f4f0 tx=0x7fc29000f9b0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 -- 172.21.15.19:0/133981914 shutdown_connections 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556412051bb0 0x55641204ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556412062730 0x556412063ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564120472f0 0x55641204e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 --2- 172.21.15.19:0/133981914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120469d0 0x556412051400 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc285ffb640 0 client.14685 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 -- 172.21.15.19:0/133981914 >> 172.21.15.19:0/133981914 conn(0x556412042f60 msgr2=0x556412048c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 -- 172.21.15.19:0/133981914 shutdown_connections 2024-09-19T04:04:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 1 -- 172.21.15.19:0/133981914 wait complete. 2024-09-19T04:04:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:58.990+0000 7fc2a724f740 20 client.14685 trim_cache size 0 max 16384 2024-09-19T04:04:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc2a724f740 1 Processor -- start 2024-09-19T04:04:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc2a724f740 1 -- start start 2024-09-19T04:04:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x556412063ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x556412043340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412041ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc2a724f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556412061ee0 con 0x5564120472f0 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc2a724f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5564120625b0 con 0x5564120469d0 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc2a724f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556412062c80 con 0x556412051bb0 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc297fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x556412043340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc297fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x556412043340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59776/0 (socket says 172.21.15.19:59776) 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.008+0000 7fc297fff640 1 -- 172.21.15.19:0/1589618966 learned_addr learned my addr 172.21.15.19:0/1589618966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc297fff640 1 -- 172.21.15.19:0/1589618966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 msgr2=0x556412041ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc29e59c640 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x556412063ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc29ed9d640 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412041ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc297fff640 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412041ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc297fff640 1 -- 172.21.15.19:0/1589618966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 msgr2=0x556412063ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc297fff640 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x556412063ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc297fff640 1 -- 172.21.15.19:0/1589618966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556412054890 con 0x5564120472f0 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc297fff640 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x556412043340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc288017fd0 tx=0x7fc2880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d7df9dfdd7e87c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc287fff640 1 -- 172.21.15.19:0/1589618966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28801e040 con 0x5564120472f0 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc287fff640 1 -- 172.21.15.19:0/1589618966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2880a3f40 con 0x5564120472f0 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.009+0000 7fc287fff640 1 -- 172.21.15.19:0/1589618966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28809ac30 con 0x5564120472f0 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- 172.21.15.19:0/1589618966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 msgr2=0x556412043340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x556412043340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc288017fd0 tx=0x7fc2880046c0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- 172.21.15.19:0/1589618966 shutdown_connections 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412041ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x556412043340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1589618966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x556412063ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- 172.21.15.19:0/1589618966 >> 172.21.15.19:0/1589618966 conn(0x556412042f60 msgr2=0x5564120417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- 172.21.15.19:0/1589618966 shutdown_connections 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- 172.21.15.19:0/1589618966 wait complete. 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- start start 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 Processor -- start 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x55641206c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x55641206c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412067530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556412063a20 con 0x5564120472f0 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556412063350 con 0x5564120469d0 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5564120640f0 con 0x556412051bb0 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc2a724f740 10 client.0 _ll_register_callbacks cb 0x7fc29000a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:04:59.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc297fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x55641206c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc297fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x55641206c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59792/0 (socket says 172.21.15.19:59792) 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.010+0000 7fc297fff640 1 -- 172.21.15.19:0/1465718406 learned_addr learned my addr 172.21.15.19:0/1465718406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc29e59c640 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x55641206c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc29ed9d640 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc297fff640 1 -- 172.21.15.19:0/1465718406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 msgr2=0x556412067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc297fff640 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412067530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc297fff640 1 -- 172.21.15.19:0/1465718406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 msgr2=0x55641206c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc297fff640 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x55641206c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc297fff640 1 -- 172.21.15.19:0/1465718406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556412067d60 con 0x5564120472f0 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc29ed9d640 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc297fff640 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x55641206c690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2880a3a10 tx=0x7fc288017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd842427a6b65f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc285ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc285ffb640 1 -- 172.21.15.19:0/1465718406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28801e070 con 0x5564120472f0 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc285ffb640 1 -- 172.21.15.19:0/1465718406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc288099370 con 0x5564120472f0 2024-09-19T04:04:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc2a724f740 1 -- 172.21.15.19:0/1465718406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2880a3f40 con 0x5564120472f0 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc285ffb640 1 -- 172.21.15.19:0/1465718406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc28809f680 con 0x5564120472f0 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc2a724f740 10 client.14691 fetch_fsmap learned FSMap version 17 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc2a724f740 1 -- 172.21.15.19:0/1465718406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5564120cc9e0 con 0x5564120472f0 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.011+0000 7fc285ffb640 1 -- 172.21.15.19:0/1465718406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2880a3f40 con 0x5564120472f0 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc285ffb640 1 -- 172.21.15.19:0/1465718406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fc2880b4140 con 0x5564120472f0 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc2a724f740 10 client.14691 fetch_fsmap finished waiting for FSMap version 17 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc2a724f740 10 client.14691 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc2a724f740 20 client.14691 populate_metadata read hostname 'smithi019' 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5564120d0e00 0x5564120cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc2a724f740 4 client.14691 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc2a724f740 1 -- 172.21.15.19:0/1465718406 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "status"}) -- 0x556411fb4b10 con 0x5564120d0e00 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc29e59c640 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5564120d0e00 0x5564120cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc285ffb640 10 client.14691 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:04:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc29e59c640 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5564120d0e00 0x5564120cfb10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93dd73208fef4c1c server_cookie=4fce477820346925 in_seq=0 out_seq=0 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc285ffb640 1 -- 172.21.15.19:0/1465718406 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556411fb4b10 con 0x5564120d0e00 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc285ffb640 10 client.14691 handle_command_reply: tid=0 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.012+0000 7fc2a724f740 1 client.14691 shutdown 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 -- 172.21.15.19:0/1465718406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 msgr2=0x55641206c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x55641206c690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2880a3a10 tx=0x7fc288017730 comp rx=0 tx=0).stop 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 -- 172.21.15.19:0/1465718406 shutdown_connections 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5564120d0e00 0x5564120cfb10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556412051bb0 0x556412067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564120472f0 0x55641206c690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc285ffb640 0 client.14691 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 --2- 172.21.15.19:0/1465718406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564120469d0 0x55641206c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 -- 172.21.15.19:0/1465718406 >> 172.21.15.19:0/1465718406 conn(0x556412042f60 msgr2=0x556412042000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 -- 172.21.15.19:0/1465718406 shutdown_connections 2024-09-19T04:04:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 1 -- 172.21.15.19:0/1465718406 wait complete. 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a724f740 20 client.14691 trim_cache size 0 max 16384 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24476, 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 18.710357017, 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 17, 2024-09-19T04:04:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-09-19T04:04:59.030 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:04:59.030 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 29.253449442000001 2024-09-19T04:04:59.030 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:04:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 -- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc28c0a32c0 msgr2=0x7fc28c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc28c0a32c0 0x7fc28c0a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc298005fc0 tx=0x7fc29802a950 comp rx=0 tx=0).stop 2024-09-19T04:04:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 -- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102570 msgr2=0x7fc2a0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:04:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102570 0x7fc2a0198bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc28800b4a0 tx=0x7fc28800b960 comp rx=0 tx=0).stop 2024-09-19T04:04:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 -- 172.21.15.19:0/1851519919 shutdown_connections 2024-09-19T04:04:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc28c0a32c0 0x7fc28c0a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a0102ef0 0x7fc2a019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102570 0x7fc2a0198bb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 --2- 172.21.15.19:0/1851519919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2a0100e20 0x7fc2a0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:04:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.013+0000 7fc2a4e76640 1 -- 172.21.15.19:0/1851519919 >> 172.21.15.19:0/1851519919 conn(0x7fc2a00fc7d0 msgr2=0x7fc2a006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:04:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.014+0000 7fc2a4e76640 1 -- 172.21.15.19:0/1851519919 shutdown_connections 2024-09-19T04:04:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:04:59.014+0000 7fc2a4e76640 1 -- 172.21.15.19:0/1851519919 wait complete. 2024-09-19T04:04:59.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:04:59 smithi019 ceph-mon[30938]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:59.681 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:04:59.681 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:04:59.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T04:04:59.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:04:59 smithi196 ceph-mon[35731]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:59.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:04:59 smithi111 ceph-mon[35365]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:04:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:00.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 -- 172.21.15.19:0/1279763548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8106a80 msgr2=0x7fafb8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1279763548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8106a80 0x7fafb8106e60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009930 tx=0x7fafa802a420 comp rx=0 tx=0).stop 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 -- 172.21.15.19:0/1279763548 shutdown_connections 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1279763548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafb8101420 0x7fafb810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1279763548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb8100a80 0x7fafb8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1279763548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8106a80 0x7fafb8106e60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 -- 172.21.15.19:0/1279763548 >> 172.21.15.19:0/1279763548 conn(0x7fafb80fc7a0 msgr2=0x7fafb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 -- 172.21.15.19:0/1279763548 shutdown_connections 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.421+0000 7fafbccfa640 1 -- 172.21.15.19:0/1279763548 wait complete. 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.422+0000 7fafbccfa640 1 Processor -- start 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.422+0000 7fafbccfa640 1 -- start start 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.422+0000 7fafbccfa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8100a80 0x7fafb80fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.422+0000 7fafbccfa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb8101420 0x7fafb80fd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.422+0000 7fafbccfa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafb8106a80 0x7fafb80fd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.422+0000 7fafbccfa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8110ea0 con 0x7fafb8100a80 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.422+0000 7fafbccfa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8110d20 con 0x7fafb8106a80 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.422+0000 7fafbccfa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8111020 con 0x7fafb8101420 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8100a80 0x7fafb80fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8100a80 0x7fafb80fcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59816/0 (socket says 172.21.15.19:59816) 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 -- 172.21.15.19:0/1092614402 learned_addr learned my addr 172.21.15.19:0/1092614402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb6d9d640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafb8106a80 0x7fafb80fd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafaffff640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb8101420 0x7fafb80fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 -- 172.21.15.19:0/1092614402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb8101420 msgr2=0x7fafb80fd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb8101420 0x7fafb80fd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 -- 172.21.15.19:0/1092614402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafb8106a80 msgr2=0x7fafb80fd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafb8106a80 0x7fafb80fd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 -- 172.21.15.19:0/1092614402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb81a6950 con 0x7fafb8100a80 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb6d9d640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafb8106a80 0x7fafb80fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafb659c640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8100a80 0x7fafb80fcd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009a50 tx=0x7fafa8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafaf7fe640 1 -- 172.21.15.19:0/1092614402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa803f030 con 0x7fafb8100a80 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafaf7fe640 1 -- 172.21.15.19:0/1092614402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa8003ce0 con 0x7fafb8100a80 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a6be0 con 0x7fafb8100a80 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafaf7fe640 1 -- 172.21.15.19:0/1092614402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8035070 con 0x7fafb8100a80 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.423+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a70c0 con 0x7fafb8100a80 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.424+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb8100ee0 con 0x7fafb8100a80 2024-09-19T04:05:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.425+0000 7fafaf7fe640 1 -- 172.21.15.19:0/1092614402 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fafa8042030 con 0x7fafb8100a80 2024-09-19T04:05:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.426+0000 7fafaf7fe640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fafa40a3240 0x7fafa40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.426+0000 7fafaffff640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fafa40a3240 0x7fafa40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.426+0000 7fafaf7fe640 1 -- 172.21.15.19:0/1092614402 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fafa80f0190 con 0x7fafb8100a80 2024-09-19T04:05:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.427+0000 7fafaffff640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fafa40a3240 0x7fafa40a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafb80fe7e0 tx=0x7fafa0005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.430+0000 7fafaf7fe640 1 -- 172.21.15.19:0/1092614402 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fafa80bcc70 con 0x7fafb8100a80 2024-09-19T04:05:00.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.649+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fafb810a000 con 0x7fafb8100a80 2024-09-19T04:05:00.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.649+0000 7fafaf7fe640 1 -- 172.21.15.19:0/1092614402 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fafa80dc070 con 0x7fafb8100a80 2024-09-19T04:05:00.651 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.651+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fafa40a3240 msgr2=0x7fafa40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.651+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fafa40a3240 0x7fafa40a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafb80fe7e0 tx=0x7fafa0005cc0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.651+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8100a80 msgr2=0x7fafb80fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.651+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8100a80 0x7fafb80fcd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009a50 tx=0x7fafa8004270 comp rx=0 tx=0).stop 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.652+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 shutdown_connections 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.652+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fafa40a3240 0x7fafa40a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.652+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafb8106a80 0x7fafb80fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.652+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb8101420 0x7fafb80fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.652+0000 7fafbccfa640 1 --2- 172.21.15.19:0/1092614402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafb8100a80 0x7fafb80fcd50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.652+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 >> 172.21.15.19:0/1092614402 conn(0x7fafb80fc7a0 msgr2=0x7fafb806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.652+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 shutdown_connections 2024-09-19T04:05:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:00.652+0000 7fafbccfa640 1 -- 172.21.15.19:0/1092614402 wait complete. 2024-09-19T04:05:01.243 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-19T04:05:01.243 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:05:01.243 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-09-19T04:05:01.243 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-19T04:05:01.243 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:05:01.243 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-19T04:05:01.244 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:05:01.244 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:05:01.244 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T04:05:01.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:01.244 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:05:01.261 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42269sec preferred_lft 42269sec 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-09-19T04:05:01.262 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-09-19T04:05:01.263 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T04:05:01.263 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:05:01.263 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:05:01.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-09-19T04:05:01.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-09-19T04:05:01.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-09-19T04:05:01.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T04:05:01.264 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:05:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:05:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:05:01.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:01.485 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:05:01.511 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-09-19T04:05:01.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:01.513 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-09-19T04:05:01.590 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-09-19T04:05:01.590 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:05:01.591 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T04:05:01.591 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T04:05:01.591 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:05:01.591 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:05:01.591 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:05:01.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T04:05:01.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T04:05:01.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T04:05:01.592 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:05:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:05:01.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:01 smithi019 ceph-mon[30938]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:05:01.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:01 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1092614402' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:05:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:01 smithi196 ceph-mon[35731]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:05:01.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:01 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1092614402' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:05:01.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:05:01.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:01.901 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-09-19T04:05:01.904 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:01 smithi111 ceph-mon[35365]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:05:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:01 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1092614402' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:05:01.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:01.917 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-09-19T04:05:01.973 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:05:01.973 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:05:01.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:05:01.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-19T04:05:01.973 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:05:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:05:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:05:02.094 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-19T04:05:02.094 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:05:02.094 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:05:02.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-19T04:05:02.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T04:05:02.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-19T04:05:02.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-19T04:05:02.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-19T04:05:02.094 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:05:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:05:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:05:02.231 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:05:02.231 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:05:02.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-09-19T04:05:02.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T04:05:02.231 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:05:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:05:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:05:02.311 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-19T04:05:02.311 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:05:02.311 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:05:02.327 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T04:05:02.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:05:02.328 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:05:02.384 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-09-19T04:05:02.451 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:05:02.508 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-09-19T04:05:02.508 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-09-19T04:05:02.508 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-09-19T04:05:02.508 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-09-19T04:05:02.508 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-09-19T04:05:02.508 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-09-19T04:05:02.508 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-09-19T04:05:02.508 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T04:05:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/5ce28c8e864c3e9f5e9abfb9f53429b7241e012e49a9fc80e37d83f313854cfb/merged 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/27e85429ecf9ee0bb040971ab725d86a8bb261f21c9bafc445b93678b0567009/merged 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/065efafe870f4a38960907c7bbb1522541bc96782fcbea4607a59a7945de8259/merged 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/9078cbb4179fc2e244aa375e655018bca3086e0585736bf15bb5439f4a1ce8dc/merged 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/8cf2184f601521399f298f7f48980f8713c626348fc1ff5b4263d18e0e3c5626/merged 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/da1ce6dec1a6f0168f1a7895d0b1bc936e239c1d30e396b0f2c517abe0892f83/merged 2024-09-19T04:05:02.510 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/960507feeb8008124461b3024bd31ba58a755382fdfeca7c9a9411e053fd1857/merged 2024-09-19T04:05:02.511 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/a2ae2f47288daa04be8ad421e726cb5f47ab7e083b654449458efaa8f4a8e3d4/merged 2024-09-19T04:05:02.511 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/3dc9d1122ea93bfc846f8abb32927410e332bebe9cd0a77daaf224e6576bb17e/merged 2024-09-19T04:05:02.511 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/1939ac7843a4da1d276d8ad872785d1388a0417c0c399921a614e236b2447b3a/merged 2024-09-19T04:05:02.511 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-09-19T04:05:02.511 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:05:02.511 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:05:02.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:02.511 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-09-19T04:05:02.564 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:05:02.564 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-19T04:05:02.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-09-19T04:05:02.632 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:05:02.689 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-09-19T04:05:02.690 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T04:05:02.691 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/5ce28c8e864c3e9f5e9abfb9f53429b7241e012e49a9fc80e37d83f313854cfb/merged 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/27e85429ecf9ee0bb040971ab725d86a8bb261f21c9bafc445b93678b0567009/merged 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/065efafe870f4a38960907c7bbb1522541bc96782fcbea4607a59a7945de8259/merged 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/9078cbb4179fc2e244aa375e655018bca3086e0585736bf15bb5439f4a1ce8dc/merged 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/8cf2184f601521399f298f7f48980f8713c626348fc1ff5b4263d18e0e3c5626/merged 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/da1ce6dec1a6f0168f1a7895d0b1bc936e239c1d30e396b0f2c517abe0892f83/merged 2024-09-19T04:05:02.692 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/960507feeb8008124461b3024bd31ba58a755382fdfeca7c9a9411e053fd1857/merged 2024-09-19T04:05:02.693 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/a2ae2f47288daa04be8ad421e726cb5f47ab7e083b654449458efaa8f4a8e3d4/merged 2024-09-19T04:05:02.693 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/3dc9d1122ea93bfc846f8abb32927410e332bebe9cd0a77daaf224e6576bb17e/merged 2024-09-19T04:05:02.693 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/1939ac7843a4da1d276d8ad872785d1388a0417c0c399921a614e236b2447b3a/merged 2024-09-19T04:05:02.693 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-09-19T04:05:02.693 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:05:02.693 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:05:02.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:02.693 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-09-19T04:05:02.721 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:2024-09-19T04:05:02.718+0000 7f4ae7a5c480 -1 init, newargv = 0x562816b39ca0 newargc=15 2024-09-19T04:05:02.721 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[74751]: starting ceph client 2024-09-19T04:05:02.726 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[74751]: starting fuse 2024-09-19T04:05:02.737 INFO:teuthology.orchestra.run.smithi019.stdout:120 2024-09-19T04:05:02.737 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> import glob 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> import re 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> import subprocess 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> def _find_admin_socket(client_name): 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> files = glob.glob(asok_path) 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> # Given a non-glob path, it better be there 2024-09-19T04:05:02.738 DEBUG:teuthology.orchestra.run.smithi019:> if "*" not in asok_path: 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> assert(len(files) == 1) 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> return files[0] 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> for f in files: 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> contents = proc_f.read() 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> if mountpoint in contents: 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> return f 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> print(_find_admin_socket("client.admin")) 2024-09-19T04:05:02.739 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-09-19T04:05:02.858 INFO:teuthology.orchestra.run.smithi019.stdout:/var/run/ceph/ceph-client.admin.74751.asok 2024-09-19T04:05:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:05:02.868 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74751.asok 2024-09-19T04:05:02.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:02.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74751.asok status 2024-09-19T04:05:03.008 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:05:03.008 INFO:teuthology.orchestra.run.smithi019.stdout: "metadata": { 2024-09-19T04:05:03.008 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:05:03.008 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:05:03.008 INFO:teuthology.orchestra.run.smithi019.stdout: "entity_id": "admin", 2024-09-19T04:05:03.008 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": "smithi019", 2024-09-19T04:05:03.008 INFO:teuthology.orchestra.run.smithi019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-19T04:05:03.008 INFO:teuthology.orchestra.run.smithi019.stdout: "pid": "74751", 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "root": "/" 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_count": 0, 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_pinned_count": 0, 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 14703, 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "inst": { 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": { 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "client", 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 14703 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-09-19T04:05:03.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 287850224 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 287850224 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "inst_str": "client.14703 192.168.0.1:0/287850224", 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "addr_str": "192.168.0.1:0/287850224", 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "inode_count": 1, 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_epoch": 17, 2024-09-19T04:05:03.010 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch": 89, 2024-09-19T04:05:03.011 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:05:03.011 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklisted": false, 2024-09-19T04:05:03.011 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs" 2024-09-19T04:05:03.011 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:05:03.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:03.016 DEBUG:teuthology.orchestra.run.smithi019:> stat --file-system '--printf=%T 2024-09-19T04:05:03.016 DEBUG:teuthology.orchestra.run.smithi019:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:05:03.033 INFO:teuthology.orchestra.run.smithi019.stdout:fuseblk 2024-09-19T04:05:03.033 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:05:03.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:03.033 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:05:03.099 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-19T04:05:03.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:05:03.208 INFO:teuthology.orchestra.run.smithi019.stdout:# file: . 2024-09-19T04:05:03.209 INFO:teuthology.orchestra.run.smithi019.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-19T04:05:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:05:03.215 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T04:05:03.215 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:05:03.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T04:05:03.533 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:03.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:03 smithi196 ceph-mon[35731]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:05:03.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:03 smithi019 ceph-mon[30938]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:05:03.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:03 smithi111 ceph-mon[35365]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:05:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.018+0000 7ff4f3943640 1 -- 172.21.15.19:0/2199750406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec101280 msgr2=0x7ff4ec10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.018+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2199750406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec101280 0x7ff4ec10df40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009900 tx=0x7ff4e002a410 comp rx=0 tx=0).stop 2024-09-19T04:05:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 -- 172.21.15.19:0/2199750406 shutdown_connections 2024-09-19T04:05:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2199750406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ec10e480 0x7ff4ec110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2199750406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec101280 0x7ff4ec10df40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2199750406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec1008d0 0x7ff4ec100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 -- 172.21.15.19:0/2199750406 >> 172.21.15.19:0/2199750406 conn(0x7ff4ec0fc7a0 msgr2=0x7ff4ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 -- 172.21.15.19:0/2199750406 shutdown_connections 2024-09-19T04:05:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 -- 172.21.15.19:0/2199750406 wait complete. 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 Processor -- start 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 -- start start 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec1008d0 0x7ff4ec19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ec101280 0x7ff4ec19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec10e480 0x7ff4ec19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec114200 con 0x7ff4ec10e480 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec114080 con 0x7ff4ec101280 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.019+0000 7ff4f3943640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec114380 con 0x7ff4ec1008d0 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec10e480 0x7ff4ec19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec10e480 0x7ff4ec19df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53588/0 (socket says 172.21.15.19:53588) 2024-09-19T04:05:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 -- 172.21.15.19:0/2019661184 learned_addr learned my addr 172.21.15.19:0/2019661184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f0ede640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ec101280 0x7ff4ec19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f16df640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec1008d0 0x7ff4ec19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 -- 172.21.15.19:0/2019661184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec1008d0 msgr2=0x7ff4ec19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec1008d0 0x7ff4ec19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 -- 172.21.15.19:0/2019661184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ec101280 msgr2=0x7ff4ec19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ec101280 0x7ff4ec19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 -- 172.21.15.19:0/2019661184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec19e6b0 con 0x7ff4ec10e480 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f16df640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec1008d0 0x7ff4ec19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f0ede640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ec101280 0x7ff4ec19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f1ee0640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec10e480 0x7ff4ec19df80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400d6a0 tx=0x7ff4e400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4ea7fc640 1 -- 172.21.15.19:0/2019661184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4017070 con 0x7ff4ec10e480 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.020+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ec19e9a0 con 0x7ff4ec10e480 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.021+0000 7ff4ea7fc640 1 -- 172.21.15.19:0/2019661184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e4009820 con 0x7ff4ec10e480 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.021+0000 7ff4ea7fc640 1 -- 172.21.15.19:0/2019661184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4004ce0 con 0x7ff4ec10e480 2024-09-19T04:05:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.021+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ec1a7310 con 0x7ff4ec10e480 2024-09-19T04:05:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.022+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ec100cb0 con 0x7ff4ec10e480 2024-09-19T04:05:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.023+0000 7ff4ea7fc640 1 -- 172.21.15.19:0/2019661184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff4e40099d0 con 0x7ff4ec10e480 2024-09-19T04:05:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.024+0000 7ff4ea7fc640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff4d40a3240 0x7ff4d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.024+0000 7ff4f16df640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff4d40a3240 0x7ff4d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.024+0000 7ff4ea7fc640 1 -- 172.21.15.19:0/2019661184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff4e40cf990 con 0x7ff4ec10e480 2024-09-19T04:05:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.024+0000 7ff4f16df640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff4d40a3240 0x7ff4d40a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009fa0 tx=0x7ff4dc009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.027+0000 7ff4ea7fc640 1 -- 172.21.15.19:0/2019661184 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff4e409c410 con 0x7ff4ec10e480 2024-09-19T04:05:04.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.213+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff4ec109610 con 0x7ff4ec10e480 2024-09-19T04:05:04.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.214+0000 7ff4ea7fc640 1 -- 172.21.15.19:0/2019661184 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7ff4e409bd10 con 0x7ff4ec10e480 2024-09-19T04:05:04.216 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1292798638 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2229567042 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2229567042 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2828287970 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/569607576 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/748113766 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3673052449 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3967546781 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3897870084 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3897870084 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-09-19T04:05:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff4d40a3240 msgr2=0x7ff4d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff4d40a3240 0x7ff4d40a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009fa0 tx=0x7ff4dc009570 comp rx=0 tx=0).stop 2024-09-19T04:05:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec10e480 msgr2=0x7ff4ec19df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec10e480 0x7ff4ec19df80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400d6a0 tx=0x7ff4e400db60 comp rx=0 tx=0).stop 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 shutdown_connections 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff4d40a3240 0x7ff4d40a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ec10e480 0x7ff4ec19df80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ec101280 0x7ff4ec19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 --2- 172.21.15.19:0/2019661184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec1008d0 0x7ff4ec19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 >> 172.21.15.19:0/2019661184 conn(0x7ff4ec0fc7a0 msgr2=0x7ff4ec0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 shutdown_connections 2024-09-19T04:05:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:04.216+0000 7ff4f3943640 1 -- 172.21.15.19:0/2019661184 wait complete. 2024-09-19T04:05:04.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:04 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2019661184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:05:04.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:04 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2019661184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:05:04.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:04 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2019661184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:05:05.031 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T04:05:05.031 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:05:05.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T04:05:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:05.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:05 smithi196 ceph-mon[35731]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T04:05:05.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:05 smithi019 ceph-mon[30938]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T04:05:05.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 -- 172.21.15.19:0/3554923431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 msgr2=0x7f4dc81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:05.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/3554923431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 0x7f4dc81029b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db802a480 comp rx=0 tx=0).stop 2024-09-19T04:05:05.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 -- 172.21.15.19:0/3554923431 shutdown_connections 2024-09-19T04:05:05.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/3554923431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dc8102ef0 0x7f4dc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:05.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/3554923431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 0x7f4dc81029b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:05.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/3554923431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dc8100e20 0x7f4dc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:05.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 -- 172.21.15.19:0/3554923431 >> 172.21.15.19:0/3554923431 conn(0x7f4dc80fc7d0 msgr2=0x7f4dc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:05.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 -- 172.21.15.19:0/3554923431 shutdown_connections 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 -- 172.21.15.19:0/3554923431 wait complete. 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.818+0000 7f4dcf33f640 1 Processor -- start 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcf33f640 1 -- start start 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcf33f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dc8100e20 0x7f4dc819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcf33f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 0x7f4dc819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcf33f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dc8102ef0 0x7f4dc819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcf33f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8110f50 con 0x7f4dc8102570 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcf33f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8110dd0 con 0x7f4dc8102ef0 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcf33f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81110d0 con 0x7f4dc8100e20 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 0x7f4dc819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 0x7f4dc819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53608/0 (socket says 172.21.15.19:53608) 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 -- 172.21.15.19:0/2034436803 learned_addr learned my addr 172.21.15.19:0/2034436803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcd8dc640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dc8102ef0 0x7f4dc819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcd0db640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dc8100e20 0x7f4dc819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 -- 172.21.15.19:0/2034436803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dc8100e20 msgr2=0x7f4dc819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dc8100e20 0x7f4dc819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 -- 172.21.15.19:0/2034436803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dc8102ef0 msgr2=0x7f4dc819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dc8102ef0 0x7f4dc819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 -- 172.21.15.19:0/2034436803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc819f6a0 con 0x7f4dc8102570 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.819+0000 7f4dcc8da640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 0x7f4dc819ac30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4db8006010 tx=0x7f4db8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.820+0000 7f4dbe7fc640 1 -- 172.21.15.19:0/2034436803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db803f070 con 0x7f4dc8102570 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.820+0000 7f4dbe7fc640 1 -- 172.21.15.19:0/2034436803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db8031e30 con 0x7f4dc8102570 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.820+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc819f930 con 0x7f4dc8102570 2024-09-19T04:05:05.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.820+0000 7f4dbe7fc640 1 -- 172.21.15.19:0/2034436803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8035030 con 0x7f4dc8102570 2024-09-19T04:05:05.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.820+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81a7270 con 0x7f4dc8102570 2024-09-19T04:05:05.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.821+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc8101200 con 0x7f4dc8102570 2024-09-19T04:05:05.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.822+0000 7f4dbe7fc640 1 -- 172.21.15.19:0/2034436803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4db8042070 con 0x7f4dc8102570 2024-09-19T04:05:05.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.822+0000 7f4dbe7fc640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4db40a32c0 0x7f4db40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:05.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.823+0000 7f4dbe7fc640 1 -- 172.21.15.19:0/2034436803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4db80f0420 con 0x7f4dc8102570 2024-09-19T04:05:05.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.823+0000 7f4dcd0db640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4db40a32c0 0x7f4db40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:05.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.823+0000 7f4dcd0db640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4db40a32c0 0x7f4db40a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005e10 tx=0x7f4dc4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:05.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:05.827+0000 7f4dbe7fc640 1 -- 172.21.15.19:0/2034436803 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4db80bcf00 con 0x7f4dc8102570 2024-09-19T04:05:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:05 smithi111 ceph-mon[35365]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T04:05:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.016+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4dc81a7440 con 0x7f4dc8102570 2024-09-19T04:05:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.017+0000 7f4dbe7fc640 1 -- 172.21.15.19:0/2034436803 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7f4db80bc800 con 0x7f4dc8102570 2024-09-19T04:05:06.019 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1292798638 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:06.019 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2229567042 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:06.019 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2229567042 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:06.019 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2828287970 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:06.019 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/569607576 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:06.019 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/748113766 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:06.020 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3673052449 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:06.020 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3967546781 2024-09-20T03:57:40.326582+0000 2024-09-19T04:05:06.020 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3897870084 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:06.020 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3897870084 2024-09-20T03:57:15.012251+0000 2024-09-19T04:05:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4db40a32c0 msgr2=0x7f4db40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4db40a32c0 0x7f4db40a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005e10 tx=0x7f4dc4005da0 comp rx=0 tx=0).stop 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 msgr2=0x7f4dc819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 0x7f4dc819ac30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4db8006010 tx=0x7f4db8003c80 comp rx=0 tx=0).stop 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 shutdown_connections 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4db40a32c0 0x7f4db40a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dc8102ef0 0x7f4dc819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc8102570 0x7f4dc819ac30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 --2- 172.21.15.19:0/2034436803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dc8100e20 0x7f4dc819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 >> 172.21.15.19:0/2034436803 conn(0x7f4dc80fc7d0 msgr2=0x7f4dc80ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 shutdown_connections 2024-09-19T04:05:06.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:06.019+0000 7f4dcf33f640 1 -- 172.21.15.19:0/2034436803 wait complete. 2024-09-19T04:05:06.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:06 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2034436803' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:05:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:06 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2034436803' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:05:06.771 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi019.front.sepia.ceph.com... 2024-09-19T04:05:06.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:06.772 DEBUG:teuthology.orchestra.run.smithi019:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:05:06.798 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[74751]: fuse finished with error 0 and tester_r 0 2024-09-19T04:05:06.800 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:05:06.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:06 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2034436803' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:05:07.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:07 smithi196 ceph-mon[35731]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T04:05:07.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:07 smithi111 ceph-mon[35365]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T04:05:07.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:07 smithi019 ceph-mon[30938]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T04:05:09.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:09 smithi196 ceph-mon[35731]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:09.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:09 smithi111 ceph-mon[35365]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:09.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:09 smithi019 ceph-mon[30938]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:11.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:11 smithi196 ceph-mon[35731]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:11.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:11 smithi111 ceph-mon[35365]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:11.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:11 smithi019 ceph-mon[30938]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:12.902 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T04:05:12.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:05:12.902 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-19T04:05:12.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:05:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:13.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:13 smithi019 ceph-mon[30938]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/1816506738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8100e20 msgr2=0x7fa7c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/1816506738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8100e20 0x7fa7c8101200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009900 tx=0x7fa7b802a480 comp rx=0 tx=0).stop 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/1816506738 shutdown_connections 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/1816506738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c8102ef0 0x7fa7c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/1816506738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7c8102570 0x7fa7c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/1816506738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8100e20 0x7fa7c8101200 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/1816506738 >> 172.21.15.19:0/1816506738 conn(0x7fa7c80fc7d0 msgr2=0x7fa7c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/1816506738 shutdown_connections 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.747+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/1816506738 wait complete. 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7cd5d3640 1 Processor -- start 2024-09-19T04:05:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7cd5d3640 1 -- start start 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7cd5d3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7c8100e20 0x7fa7c8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7cd5d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8102570 0x7fa7c81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7cd5d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c8102ef0 0x7fa7c819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7cd5d3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c8110f80 con 0x7fa7c8102570 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7cd5d3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c8110e00 con 0x7fa7c8100e20 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7cd5d3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c8111100 con 0x7fa7c8102ef0 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7c67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8102570 0x7fa7c81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7c67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8102570 0x7fa7c81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58530/0 (socket says 172.21.15.19:58530) 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.748+0000 7fa7c67fc640 1 -- 172.21.15.19:0/3799162373 learned_addr learned my addr 172.21.15.19:0/3799162373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c77fe640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c8102ef0 0x7fa7c819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c6ffd640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7c8100e20 0x7fa7c8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c67fc640 1 -- 172.21.15.19:0/3799162373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c8102ef0 msgr2=0x7fa7c819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c67fc640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c8102ef0 0x7fa7c819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c67fc640 1 -- 172.21.15.19:0/3799162373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7c8100e20 msgr2=0x7fa7c8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c67fc640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7c8100e20 0x7fa7c8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c67fc640 1 -- 172.21.15.19:0/3799162373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c819d410 con 0x7fa7c8102570 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c6ffd640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7c8100e20 0x7fa7c8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7c67fc640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8102570 0x7fa7c81989a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c920 tx=0x7fa7bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7affff640 1 -- 172.21.15.19:0/3799162373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc007c70 con 0x7fa7c8102570 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7affff640 1 -- 172.21.15.19:0/3799162373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7bc007e10 con 0x7fa7c8102570 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c819d700 con 0x7fa7c8102570 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7affff640 1 -- 172.21.15.19:0/3799162373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc018890 con 0x7fa7c8102570 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.749+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c81a4fe0 con 0x7fa7c8102570 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.750+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c8101200 con 0x7fa7c8102570 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.752+0000 7fa7affff640 1 -- 172.21.15.19:0/3799162373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa7bc018a70 con 0x7fa7c8102570 2024-09-19T04:05:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.752+0000 7fa7affff640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa7b00a3240 0x7fa7b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.752+0000 7fa7affff640 1 -- 172.21.15.19:0/3799162373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa7bc0cfca0 con 0x7fa7c8102570 2024-09-19T04:05:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.752+0000 7fa7c6ffd640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa7b00a3240 0x7fa7b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.753+0000 7fa7c6ffd640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa7b00a3240 0x7fa7b00a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8005ab0 tx=0x7fa7b8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.756+0000 7fa7affff640 1 -- 172.21.15.19:0/3799162373 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa7bc014030 con 0x7fa7c8102570 2024-09-19T04:05:13.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:13 smithi111 ceph-mon[35365]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.935+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7c810a0a0 con 0x7fa7c8102570 2024-09-19T04:05:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.937+0000 7fa7affff640 1 -- 172.21.15.19:0/3799162373 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24188 (secure 0 0 0) 0x7fa7bc09c780 con 0x7fa7c8102570 2024-09-19T04:05:13.939 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:13.939 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:04:48.953123+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:05:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa7b00a3240 msgr2=0x7fa7b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa7b00a3240 0x7fa7b00a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8005ab0 tx=0x7fa7b8005a20 comp rx=0 tx=0).stop 2024-09-19T04:05:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8102570 msgr2=0x7fa7c81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8102570 0x7fa7c81989a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c920 tx=0x7fa7bc00cde0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 shutdown_connections 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa7b00a3240 0x7fa7b00a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c8102ef0 0x7fa7c819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c8102570 0x7fa7c81989a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 --2- 172.21.15.19:0/3799162373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7c8100e20 0x7fa7c8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 >> 172.21.15.19:0/3799162373 conn(0x7fa7c80fc7d0 msgr2=0x7fa7c806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 shutdown_connections 2024-09-19T04:05:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:13.939+0000 7fa7cd5d3640 1 -- 172.21.15.19:0/3799162373 wait complete. 2024-09-19T04:05:13.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:13 smithi196 ceph-mon[35731]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:14.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:14 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3799162373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:14.680 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:05:14.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:14 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3799162373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:14.937 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:14.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:14 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3799162373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 -- 172.21.15.19:0/4053258205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8102790 msgr2=0x7f34c81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 --2- 172.21.15.19:0/4053258205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8102790 0x7f34c81067e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009900 tx=0x7f34b802a410 comp rx=0 tx=0).stop 2024-09-19T04:05:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 -- 172.21.15.19:0/4053258205 shutdown_connections 2024-09-19T04:05:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 --2- 172.21.15.19:0/4053258205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8106f10 0x7f34c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 --2- 172.21.15.19:0/4053258205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8102790 0x7f34c81067e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 --2- 172.21.15.19:0/4053258205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34c8101de0 0x7f34c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 -- 172.21.15.19:0/4053258205 >> 172.21.15.19:0/4053258205 conn(0x7f34c80fd8c0 msgr2=0x7f34c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 -- 172.21.15.19:0/4053258205 shutdown_connections 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.393+0000 7f34d00cf640 1 -- 172.21.15.19:0/4053258205 wait complete. 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34d00cf640 1 Processor -- start 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34d00cf640 1 -- start start 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34d00cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8101de0 0x7f34c80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34d00cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34c8102790 0x7f34c8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34d00cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8106f10 0x7f34c8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34d00cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8111fc0 con 0x7f34c8106f10 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34d00cf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8111e40 con 0x7f34c8102790 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34d00cf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8112140 con 0x7f34c8101de0 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34ce66c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8106f10 0x7f34c8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34ce66c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8106f10 0x7f34c8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58544/0 (socket says 172.21.15.19:58544) 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.394+0000 7f34ce66c640 1 -- 172.21.15.19:0/1713168292 learned_addr learned my addr 172.21.15.19:0/1713168292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34cde6b640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8101de0 0x7f34c80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34cd66a640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34c8102790 0x7f34c8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34ce66c640 1 -- 172.21.15.19:0/1713168292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8101de0 msgr2=0x7f34c80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34ce66c640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8101de0 0x7f34c80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34ce66c640 1 -- 172.21.15.19:0/1713168292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34c8102790 msgr2=0x7f34c8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34ce66c640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34c8102790 0x7f34c8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34ce66c640 1 -- 172.21.15.19:0/1713168292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c81a6940 con 0x7f34c8106f10 2024-09-19T04:05:15.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34ce66c640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8106f10 0x7f34c8074f70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f34c000eaa0 tx=0x7f34c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:15.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34beffd640 1 -- 172.21.15.19:0/1713168292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c000cdb0 con 0x7f34c8106f10 2024-09-19T04:05:15.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34beffd640 1 -- 172.21.15.19:0/1713168292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c0005020 con 0x7f34c8106f10 2024-09-19T04:05:15.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c81a6c30 con 0x7f34c8106f10 2024-09-19T04:05:15.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34beffd640 1 -- 172.21.15.19:0/1713168292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00116b0 con 0x7f34c8106f10 2024-09-19T04:05:15.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.395+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c81a7190 con 0x7f34c8106f10 2024-09-19T04:05:15.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.396+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c81021c0 con 0x7f34c8106f10 2024-09-19T04:05:15.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.397+0000 7f34beffd640 1 -- 172.21.15.19:0/1713168292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f34c0005560 con 0x7f34c8106f10 2024-09-19T04:05:15.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.398+0000 7f34beffd640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34b40a32c0 0x7f34b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:15.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.398+0000 7f34cde6b640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34b40a32c0 0x7f34b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:15.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.398+0000 7f34beffd640 1 -- 172.21.15.19:0/1713168292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f34c00cfbe0 con 0x7f34c8106f10 2024-09-19T04:05:15.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.399+0000 7f34cde6b640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34b40a32c0 0x7f34b40a5780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009bd0 tx=0x7f34c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:15.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.402+0000 7f34beffd640 1 -- 172.21.15.19:0/1713168292 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34c0016030 con 0x7f34c8106f10 2024-09-19T04:05:15.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.582+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34c810a150 con 0x7f34c8106f10 2024-09-19T04:05:15.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:15 smithi019 ceph-mon[30938]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.584+0000 7f34beffd640 1 -- 172.21.15.19:0/1713168292 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24188 (secure 0 0 0) 0x7f34c009c070 con 0x7f34c8106f10 2024-09-19T04:05:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:15.587 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:04:48.953123+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:05:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.586+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34b40a32c0 msgr2=0x7f34b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.586+0000 7f34d00cf640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34b40a32c0 0x7f34b40a5780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009bd0 tx=0x7f34c4005f90 comp rx=0 tx=0).stop 2024-09-19T04:05:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.586+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8106f10 msgr2=0x7f34c8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.586+0000 7f34d00cf640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8106f10 0x7f34c8074f70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f34c000eaa0 tx=0x7f34c000ef60 comp rx=0 tx=0).stop 2024-09-19T04:05:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.587+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 shutdown_connections 2024-09-19T04:05:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.587+0000 7f34d00cf640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f34b40a32c0 0x7f34b40a5780 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.587+0000 7f34d00cf640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c8106f10 0x7f34c8074f70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.587+0000 7f34d00cf640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34c8102790 0x7f34c8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.587+0000 7f34d00cf640 1 --2- 172.21.15.19:0/1713168292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8101de0 0x7f34c80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:15.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.587+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 >> 172.21.15.19:0/1713168292 conn(0x7f34c80fd8c0 msgr2=0x7f34c80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:15.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.587+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 shutdown_connections 2024-09-19T04:05:15.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:15.587+0000 7f34d00cf640 1 -- 172.21.15.19:0/1713168292 wait complete. 2024-09-19T04:05:15.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:15 smithi111 ceph-mon[35365]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:15.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:15 smithi196 ceph-mon[35731]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T04:05:16.372 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:05:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:16.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:16 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1713168292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:16.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:16 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1713168292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:16.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:16 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1713168292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 -- 172.21.15.19:0/151254612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 msgr2=0x7f4e2010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 --2- 172.21.15.19:0/151254612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 0x7f4e2010e610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009ca0 tx=0x7f4e1802a950 comp rx=0 tx=0).stop 2024-09-19T04:05:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 -- 172.21.15.19:0/151254612 shutdown_connections 2024-09-19T04:05:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 --2- 172.21.15.19:0/151254612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 0x7f4e2010e610 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 --2- 172.21.15.19:0/151254612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e20103700 0x7f4e2010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 --2- 172.21.15.19:0/151254612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e20102de0 0x7f4e201031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 -- 172.21.15.19:0/151254612 >> 172.21.15.19:0/151254612 conn(0x7f4e200fc7a0 msgr2=0x7f4e200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 -- 172.21.15.19:0/151254612 shutdown_connections 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.078+0000 7f4e2777e640 1 -- 172.21.15.19:0/151254612 wait complete. 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e2777e640 1 Processor -- start 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e2777e640 1 -- start start 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e2777e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e20102de0 0x7f4e20100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e2777e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e20103700 0x7f4e20100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e2777e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 0x7f4e201014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e2777e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e20111fb0 con 0x7f4e2010c180 2024-09-19T04:05:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e2777e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e20111e30 con 0x7f4e20102de0 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e2777e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e20112130 con 0x7f4e20103700 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.079+0000 7f4e25d1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 0x7f4e201014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e25d1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 0x7f4e201014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58558/0 (socket says 172.21.15.19:58558) 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e25d1b640 1 -- 172.21.15.19:0/119653783 learned_addr learned my addr 172.21.15.19:0/119653783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e2551a640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e20102de0 0x7f4e20100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e24d19640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e20103700 0x7f4e20100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e25d1b640 1 -- 172.21.15.19:0/119653783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e20103700 msgr2=0x7f4e20100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e25d1b640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e20103700 0x7f4e20100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e25d1b640 1 -- 172.21.15.19:0/119653783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e20102de0 msgr2=0x7f4e20100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e25d1b640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e20102de0 0x7f4e20100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e25d1b640 1 -- 172.21.15.19:0/119653783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e201a68b0 con 0x7f4e2010c180 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e24d19640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e20103700 0x7f4e20100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:05:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e25d1b640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 0x7f4e201014c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e18006010 tx=0x7f4e1802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e167fc640 1 -- 172.21.15.19:0/119653783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18034040 con 0x7f4e2010c180 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e167fc640 1 -- 172.21.15.19:0/119653783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e18003e00 con 0x7f4e2010c180 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e201a6b40 con 0x7f4e2010c180 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e167fc640 1 -- 172.21.15.19:0/119653783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e180456e0 con 0x7f4e2010c180 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.080+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e201a7020 con 0x7f4e2010c180 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.081+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e201031c0 con 0x7f4e2010c180 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.082+0000 7f4e167fc640 1 -- 172.21.15.19:0/119653783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4e18037020 con 0x7f4e2010c180 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.083+0000 7f4e167fc640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e0c0a3240 0x7f4e0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.083+0000 7f4e2551a640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e0c0a3240 0x7f4e0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.083+0000 7f4e167fc640 1 -- 172.21.15.19:0/119653783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4e180f0ce0 con 0x7f4e2010c180 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.084+0000 7f4e2551a640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e0c0a3240 0x7f4e0c0a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009890 tx=0x7f4e1c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.087+0000 7f4e167fc640 1 -- 172.21.15.19:0/119653783 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e180bd7c0 con 0x7f4e2010c180 2024-09-19T04:05:17.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.303+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4e201a7320 con 0x7f4e2010c180 2024-09-19T04:05:17.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.476+0000 7f4e167fc640 1 -- 172.21.15.19:0/119653783 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f4e180bd0c0 con 0x7f4e2010c180 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e0c0a3240 msgr2=0x7f4e0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e0c0a3240 0x7f4e0c0a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009890 tx=0x7f4e1c008040 comp rx=0 tx=0).stop 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 msgr2=0x7f4e201014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 0x7f4e201014c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e18006010 tx=0x7f4e1802f710 comp rx=0 tx=0).stop 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 shutdown_connections 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e0c0a3240 0x7f4e0c0a5700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e2010c180 0x7f4e201014c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e20103700 0x7f4e20100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 --2- 172.21.15.19:0/119653783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e20102de0 0x7f4e20100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.478+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 >> 172.21.15.19:0/119653783 conn(0x7f4e200fc7a0 msgr2=0x7f4e200fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.479+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 shutdown_connections 2024-09-19T04:05:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:17.479+0000 7f4e2777e640 1 -- 172.21.15.19:0/119653783 wait complete. 2024-09-19T04:05:17.791 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:17 smithi019 ceph-mon[30938]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T04:05:17.791 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:17 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/119653783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:05:17.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:17 smithi111 ceph-mon[35365]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T04:05:17.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:17 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/119653783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:05:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:17 smithi196 ceph-mon[35731]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T04:05:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:17 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/119653783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T04:05:18.216 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-19T04:05:18.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T04:05:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:18.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:18 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/119653783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:05:18.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:18 smithi019 ceph-mon[30938]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:05:18.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:18 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/119653783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:05:18.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:18 smithi111 ceph-mon[35365]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:05:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.937+0000 7f21901e9640 1 -- 172.21.15.19:0/3527552576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2188101280 msgr2=0x7f218810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.937+0000 7f21901e9640 1 --2- 172.21.15.19:0/3527552576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2188101280 0x7f218810df40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2178009900 tx=0x7f217802a410 comp rx=0 tx=0).stop 2024-09-19T04:05:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 -- 172.21.15.19:0/3527552576 shutdown_connections 2024-09-19T04:05:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 --2- 172.21.15.19:0/3527552576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f218810e480 0x7f2188110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 --2- 172.21.15.19:0/3527552576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2188101280 0x7f218810df40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 --2- 172.21.15.19:0/3527552576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21881008d0 0x7f2188100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 -- 172.21.15.19:0/3527552576 >> 172.21.15.19:0/3527552576 conn(0x7f21880fc7a0 msgr2=0x7f21880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 -- 172.21.15.19:0/3527552576 shutdown_connections 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 -- 172.21.15.19:0/3527552576 wait complete. 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 Processor -- start 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 -- start start 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21881008d0 0x7f218819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2188101280 0x7f218819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218810e480 0x7f218819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21881141a0 con 0x7f218810e480 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2188114020 con 0x7f21881008d0 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.938+0000 7f21901e9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2188114320 con 0x7f2188101280 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218810e480 0x7f218819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218810e480 0x7f218819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58576/0 (socket says 172.21.15.19:58576) 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 -- 172.21.15.19:0/1559026208 learned_addr learned my addr 172.21.15.19:0/1559026208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218df85640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21881008d0 0x7f218819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218d784640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2188101280 0x7f218819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 -- 172.21.15.19:0/1559026208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2188101280 msgr2=0x7f218819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2188101280 0x7f218819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 -- 172.21.15.19:0/1559026208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21881008d0 msgr2=0x7f218819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21881008d0 0x7f218819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 -- 172.21.15.19:0/1559026208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218819f650 con 0x7f218810e480 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f218e786640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218810e480 0x7f218819ef70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2180009fd0 tx=0x7f218000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f217effd640 1 -- 172.21.15.19:0/1559026208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180004d20 con 0x7f218810e480 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f217effd640 1 -- 172.21.15.19:0/1559026208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2180005020 con 0x7f218810e480 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218819f940 con 0x7f218810e480 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f217effd640 1 -- 172.21.15.19:0/1559026208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21800116d0 con 0x7f218810e480 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.939+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21881a7220 con 0x7f218810e480 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.940+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2188100cb0 con 0x7f218810e480 2024-09-19T04:05:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.942+0000 7f217effd640 1 -- 172.21.15.19:0/1559026208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f21800051d0 con 0x7f218810e480 2024-09-19T04:05:18.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.942+0000 7f217effd640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f21740a32c0 0x7f21740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:18.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.943+0000 7f218df85640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f21740a32c0 0x7f21740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:18.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.943+0000 7f217effd640 1 -- 172.21.15.19:0/1559026208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f21800cff10 con 0x7f218810e480 2024-09-19T04:05:18.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.943+0000 7f218df85640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f21740a32c0 0x7f21740a5780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2184009bd0 tx=0x7f2184005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:18.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:18.947+0000 7f217effd640 1 -- 172.21.15.19:0/1559026208 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2180016030 con 0x7f218810e480 2024-09-19T04:05:18.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:18 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/119653783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T04:05:18.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:18 smithi196 ceph-mon[35731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:05:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.162+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f218802d7c0 con 0x7f218810e480 2024-09-19T04:05:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.483+0000 7f217effd640 1 -- 172.21.15.19:0/1559026208 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f218009c3a0 con 0x7f218810e480 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.485+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f21740a32c0 msgr2=0x7f21740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.485+0000 7f21901e9640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f21740a32c0 0x7f21740a5780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2184009bd0 tx=0x7f2184005f90 comp rx=0 tx=0).stop 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.485+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218810e480 msgr2=0x7f218819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.485+0000 7f21901e9640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218810e480 0x7f218819ef70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2180009fd0 tx=0x7f218000ee60 comp rx=0 tx=0).stop 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.485+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 shutdown_connections 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.485+0000 7f21901e9640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f21740a32c0 0x7f21740a5780 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.486+0000 7f21901e9640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218810e480 0x7f218819ef70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.486+0000 7f21901e9640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2188101280 0x7f218819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.486+0000 7f21901e9640 1 --2- 172.21.15.19:0/1559026208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21881008d0 0x7f218819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.486+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 >> 172.21.15.19:0/1559026208 conn(0x7f21880fc7a0 msgr2=0x7f21880feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.486+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 shutdown_connections 2024-09-19T04:05:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:19.486+0000 7f21901e9640 1 -- 172.21.15.19:0/1559026208 wait complete. 2024-09-19T04:05:19.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:19 smithi019 ceph-mon[30938]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T04:05:19.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:19 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1559026208' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:05:19.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:19 smithi111 ceph-mon[35365]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T04:05:19.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:19 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1559026208' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:05:19.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:19 smithi196 ceph-mon[35731]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T04:05:19.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:19 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1559026208' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T04:05:20.285 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T04:05:20.554 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:20.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:20 smithi019 ceph-mon[30938]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:05:20.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:20 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1559026208' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:05:20.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:20 smithi019 ceph-mon[30938]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:05:20.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:20 smithi019 ceph-mon[30938]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:05:20.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:20 smithi019 ceph-mon[30938]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:05:20.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:20 smithi019 ceph-mon[30938]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:05:20.806 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:20 smithi111 ceph-mon[35365]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:05:20.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:20 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1559026208' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:05:20.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:20 smithi111 ceph-mon[35365]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:05:20.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:20 smithi111 ceph-mon[35365]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:05:20.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:20 smithi111 ceph-mon[35365]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:05:20.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:20 smithi111 ceph-mon[35365]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:05:20.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:20 smithi196 ceph-mon[35731]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T04:05:20.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:20 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1559026208' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T04:05:20.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:20 smithi196 ceph-mon[35731]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T04:05:20.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:20 smithi196 ceph-mon[35731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T04:05:20.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:20 smithi196 ceph-mon[35731]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T04:05:20.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:20 smithi196 ceph-mon[35731]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T04:05:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 -- 172.21.15.19:0/2506505374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8101420 msgr2=0x7fddc810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 --2- 172.21.15.19:0/2506505374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8101420 0x7fddc810d950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009cd0 tx=0x7fddc002a8e0 comp rx=0 tx=0).stop 2024-09-19T04:05:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 -- 172.21.15.19:0/2506505374 shutdown_connections 2024-09-19T04:05:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 --2- 172.21.15.19:0/2506505374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8101420 0x7fddc810d950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 --2- 172.21.15.19:0/2506505374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc8100a80 0x7fddc8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 --2- 172.21.15.19:0/2506505374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddc8106a80 0x7fddc8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 -- 172.21.15.19:0/2506505374 >> 172.21.15.19:0/2506505374 conn(0x7fddc80fc7a0 msgr2=0x7fddc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 -- 172.21.15.19:0/2506505374 shutdown_connections 2024-09-19T04:05:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.996+0000 7fddccd9c640 1 -- 172.21.15.19:0/2506505374 wait complete. 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddccd9c640 1 Processor -- start 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddccd9c640 1 -- start start 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddccd9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8100a80 0x7fddc8197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddccd9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddc8101420 0x7fddc8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddccd9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc8106a80 0x7fddc819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddccd9c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8110e80 con 0x7fddc8100a80 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddccd9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8110d00 con 0x7fddc8106a80 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddccd9c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8111000 con 0x7fddc8101420 2024-09-19T04:05:21.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddc659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8100a80 0x7fddc8197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddc659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8100a80 0x7fddc8197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58582/0 (socket says 172.21.15.19:58582) 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddc659c640 1 -- 172.21.15.19:0/367692858 learned_addr learned my addr 172.21.15.19:0/367692858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.997+0000 7fddc5d9b640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddc8101420 0x7fddc8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddc6d9d640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc8106a80 0x7fddc819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddc659c640 1 -- 172.21.15.19:0/367692858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddc8101420 msgr2=0x7fddc8197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddc659c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddc8101420 0x7fddc8197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddc659c640 1 -- 172.21.15.19:0/367692858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc8106a80 msgr2=0x7fddc819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddc659c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc8106a80 0x7fddc819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddc659c640 1 -- 172.21.15.19:0/367692858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc819c2d0 con 0x7fddc8100a80 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddc659c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8100a80 0x7fddc8197320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fddb800d770 tx=0x7fddb800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddb77fe640 1 -- 172.21.15.19:0/367692858 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8017070 con 0x7fddc8100a80 2024-09-19T04:05:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc819c5c0 con 0x7fddc8100a80 2024-09-19T04:05:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddb77fe640 1 -- 172.21.15.19:0/367692858 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8005500 con 0x7fddc8100a80 2024-09-19T04:05:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddb77fe640 1 -- 172.21.15.19:0/367692858 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8004e00 con 0x7fddc8100a80 2024-09-19T04:05:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.998+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc81a3ea0 con 0x7fddc8100a80 2024-09-19T04:05:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:20.999+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc8100ee0 con 0x7fddc8100a80 2024-09-19T04:05:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.000+0000 7fddb77fe640 1 -- 172.21.15.19:0/367692858 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fddb8004fa0 con 0x7fddc8100a80 2024-09-19T04:05:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.001+0000 7fddb77fe640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddb00a3240 0x7fddb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.001+0000 7fddc5d9b640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddb00a3240 0x7fddb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.001+0000 7fddb77fe640 1 -- 172.21.15.19:0/367692858 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fddb80cf520 con 0x7fddc8100a80 2024-09-19T04:05:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.002+0000 7fddc5d9b640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddb00a3240 0x7fddb00a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fddc80fed60 tx=0x7fddbc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.005+0000 7fddb77fe640 1 -- 172.21.15.19:0/367692858 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fddb809c000 con 0x7fddc8100a80 2024-09-19T04:05:21.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.220+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fddc8198680 con 0x7fddc8100a80 2024-09-19T04:05:21.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.499+0000 7fddb77fe640 1 -- 172.21.15.19:0/367692858 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fddb809b900 con 0x7fddc8100a80 2024-09-19T04:05:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.501+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddb00a3240 msgr2=0x7fddb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.501+0000 7fddccd9c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddb00a3240 0x7fddb00a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fddc80fed60 tx=0x7fddbc00b390 comp rx=0 tx=0).stop 2024-09-19T04:05:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.501+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8100a80 msgr2=0x7fddc8197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.501+0000 7fddccd9c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8100a80 0x7fddc8197320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fddb800d770 tx=0x7fddb800dc30 comp rx=0 tx=0).stop 2024-09-19T04:05:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.502+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 shutdown_connections 2024-09-19T04:05:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.502+0000 7fddccd9c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddb00a3240 0x7fddb00a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.502+0000 7fddccd9c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc8106a80 0x7fddc819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.502+0000 7fddccd9c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddc8101420 0x7fddc8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.502+0000 7fddccd9c640 1 --2- 172.21.15.19:0/367692858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc8100a80 0x7fddc8197320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.502+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 >> 172.21.15.19:0/367692858 conn(0x7fddc80fc7a0 msgr2=0x7fddc80fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.502+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 shutdown_connections 2024-09-19T04:05:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:21.502+0000 7fddccd9c640 1 -- 172.21.15.19:0/367692858 wait complete. 2024-09-19T04:05:21.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:21 smithi019 ceph-mon[30938]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-09-19T04:05:21.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:21 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] up:active 2024-09-19T04:05:21.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:21 smithi019 ceph-mon[30938]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:05:21.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:21 smithi019 ceph-mon[30938]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:05:21.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:21 smithi019 ceph-mon[30938]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:05:21.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:21 smithi019 ceph-mon[30938]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:05:21.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:21 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/367692858' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:05:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:21 smithi111 ceph-mon[35365]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-09-19T04:05:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:21 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] up:active 2024-09-19T04:05:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:21 smithi111 ceph-mon[35365]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:05:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:21 smithi111 ceph-mon[35365]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:05:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:21 smithi111 ceph-mon[35365]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:05:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:21 smithi111 ceph-mon[35365]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:05:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:21 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/367692858' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:05:21.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:21 smithi196 ceph-mon[35731]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-09-19T04:05:21.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:21 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] up:active 2024-09-19T04:05:21.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:21 smithi196 ceph-mon[35731]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T04:05:21.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:21 smithi196 ceph-mon[35731]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T04:05:21.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:21 smithi196 ceph-mon[35731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T04:05:21.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:21 smithi196 ceph-mon[35731]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T04:05:21.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:21 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/367692858' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T04:05:22.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T04:05:22.569 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:22.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:22 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/367692858' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:05:22.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:22 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] up:active 2024-09-19T04:05:22.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:22 smithi019 ceph-mon[30938]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:05:22.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:22 smithi019 ceph-mon[30938]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:05:22.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:22 smithi019 ceph-mon[30938]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-19T04:05:22.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:22 smithi019 ceph-mon[30938]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T04:05:22.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:22 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/367692858' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:05:22.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:22 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] up:active 2024-09-19T04:05:22.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:22 smithi196 ceph-mon[35731]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:05:22.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:22 smithi196 ceph-mon[35731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:05:22.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:22 smithi196 ceph-mon[35731]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-19T04:05:22.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:22 smithi196 ceph-mon[35731]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T04:05:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:22 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/367692858' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T04:05:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:22 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] up:active 2024-09-19T04:05:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:22 smithi111 ceph-mon[35365]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T04:05:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:22 smithi111 ceph-mon[35365]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T04:05:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:22 smithi111 ceph-mon[35365]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-19T04:05:22.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:22 smithi111 ceph-mon[35365]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 -- 172.21.15.19:0/4191305824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8764102d40 msgr2=0x7f8764103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 --2- 172.21.15.19:0/4191305824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8764102d40 0x7f8764103120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8754009900 tx=0x7f875402a480 comp rx=0 tx=0).stop 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 -- 172.21.15.19:0/4191305824 shutdown_connections 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 --2- 172.21.15.19:0/4191305824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f876410c170 0x7f876410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 --2- 172.21.15.19:0/4191305824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8764103660 0x7f876410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 --2- 172.21.15.19:0/4191305824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8764102d40 0x7f8764103120 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 -- 172.21.15.19:0/4191305824 >> 172.21.15.19:0/4191305824 conn(0x7f87640fc7a0 msgr2=0x7f87640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 -- 172.21.15.19:0/4191305824 shutdown_connections 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.056+0000 7f8769eaf640 1 -- 172.21.15.19:0/4191305824 wait complete. 2024-09-19T04:05:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8769eaf640 1 Processor -- start 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8769eaf640 1 -- start start 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8769eaf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8764102d40 0x7f87641009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8769eaf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8764103660 0x7f8764100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8769eaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f876410c170 0x7f8764101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8769eaf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8764111fb0 con 0x7f876410c170 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8769eaf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8764111e30 con 0x7f8764103660 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8769eaf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8764112130 con 0x7f8764102d40 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8763fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f876410c170 0x7f8764101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8763fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f876410c170 0x7f8764101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52758/0 (socket says 172.21.15.19:52758) 2024-09-19T04:05:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8763fff640 1 -- 172.21.15.19:0/304032003 learned_addr learned my addr 172.21.15.19:0/304032003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8762ffd640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8764103660 0x7f8764100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f87637fe640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8764102d40 0x7f87641009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8763fff640 1 -- 172.21.15.19:0/304032003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8764102d40 msgr2=0x7f87641009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8763fff640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8764102d40 0x7f87641009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.057+0000 7f8763fff640 1 -- 172.21.15.19:0/304032003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8764103660 msgr2=0x7f8764100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8763fff640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8764103660 0x7f8764100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8763fff640 1 -- 172.21.15.19:0/304032003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87641a68f0 con 0x7f876410c170 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8762ffd640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8764103660 0x7f8764100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8763fff640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f876410c170 0x7f8764101420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f875c00cce0 tx=0x7f875c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8760ff9640 1 -- 172.21.15.19:0/304032003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c00f450 con 0x7f876410c170 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8760ff9640 1 -- 172.21.15.19:0/304032003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875c00fad0 con 0x7f876410c170 2024-09-19T04:05:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87641a6be0 con 0x7f876410c170 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8760ff9640 1 -- 172.21.15.19:0/304032003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c018c10 con 0x7f876410c170 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.058+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87641a7120 con 0x7f876410c170 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.059+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8764103120 con 0x7f876410c170 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.060+0000 7f8760ff9640 1 -- 172.21.15.19:0/304032003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f875c00f5f0 con 0x7f876410c170 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.061+0000 7f8760ff9640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f874c0a32c0 0x7f874c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.061+0000 7f87637fe640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f874c0a32c0 0x7f874c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.061+0000 7f8760ff9640 1 -- 172.21.15.19:0/304032003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f875c0d0240 con 0x7f876410c170 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.062+0000 7f87637fe640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f874c0a32c0 0x7f874c0a5780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f87540096c0 tx=0x7f8754000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.066+0000 7f8760ff9640 1 -- 172.21.15.19:0/304032003 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f875c014030 con 0x7f876410c170 2024-09-19T04:05:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.277+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8764107210 con 0x7f876410c170 2024-09-19T04:05:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.603+0000 7f8760ff9640 1 -- 172.21.15.19:0/304032003 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f875c09cd20 con 0x7f876410c170 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] up:active 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: Cluster is now healthy 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] up:active 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:05:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:23 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/304032003' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:05:23.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.605+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f874c0a32c0 msgr2=0x7f874c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:23.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.605+0000 7f8769eaf640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f874c0a32c0 0x7f874c0a5780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f87540096c0 tx=0x7f8754000bc0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.605+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f876410c170 msgr2=0x7f8764101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:23.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.605+0000 7f8769eaf640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f876410c170 0x7f8764101420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f875c00cce0 tx=0x7f875c00a620 comp rx=0 tx=0).stop 2024-09-19T04:05:23.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.605+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 shutdown_connections 2024-09-19T04:05:23.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.605+0000 7f8769eaf640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f874c0a32c0 0x7f874c0a5780 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.605+0000 7f8769eaf640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f876410c170 0x7f8764101420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.605+0000 7f8769eaf640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8764103660 0x7f8764100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.606+0000 7f8769eaf640 1 --2- 172.21.15.19:0/304032003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8764102d40 0x7f87641009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.606+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 >> 172.21.15.19:0/304032003 conn(0x7f87640fc7a0 msgr2=0x7f87640fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.606+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 shutdown_connections 2024-09-19T04:05:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:23.606+0000 7f8769eaf640 1 -- 172.21.15.19:0/304032003 wait complete. 2024-09-19T04:05:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-09-19T04:05:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] up:active 2024-09-19T04:05:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:05:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:05:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: Cluster is now healthy 2024-09-19T04:05:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-19T04:05:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:05:23.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:05:23.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] up:active 2024-09-19T04:05:23.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:05:23.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:23 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/304032003' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:05:23.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-09-19T04:05:23.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] up:active 2024-09-19T04:05:23.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T04:05:23.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T04:05:23.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: Cluster is now healthy 2024-09-19T04:05:23.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-09-19T04:05:23.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T04:05:23.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T04:05:23.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] up:active 2024-09-19T04:05:23.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:05:23.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:23 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/304032003' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T04:05:24.248 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-19T04:05:24.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T04:05:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:24.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:24 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/304032003' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:05:24.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:24 smithi111 ceph-mon[35365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:05:24.912 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:24 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/304032003' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:05:24.912 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:24 smithi019 ceph-mon[30938]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:05:24.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:24 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/304032003' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T04:05:24.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:24 smithi196 ceph-mon[35731]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:05:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 -- 172.21.15.19:0/1765384531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d010c180 msgr2=0x7f73d010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1765384531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d010c180 0x7f73d010e600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009ca0 tx=0x7f73c802a950 comp rx=0 tx=0).stop 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 -- 172.21.15.19:0/1765384531 shutdown_connections 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1765384531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d010c180 0x7f73d010e600 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1765384531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d01036c0 0x7f73d010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1765384531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d0102da0 0x7f73d0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 -- 172.21.15.19:0/1765384531 >> 172.21.15.19:0/1765384531 conn(0x7f73d00fc7a0 msgr2=0x7f73d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 -- 172.21.15.19:0/1765384531 shutdown_connections 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 -- 172.21.15.19:0/1765384531 wait complete. 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.996+0000 7f73d78fe640 1 Processor -- start 2024-09-19T04:05:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d78fe640 1 -- start start 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d78fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d0102da0 0x7f73d0100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d78fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d01036c0 0x7f73d0100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d78fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d010c180 0x7f73d0101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d78fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0111fa0 con 0x7f73d01036c0 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d78fe640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0111e20 con 0x7f73d0102da0 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d78fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0112120 con 0x7f73d010c180 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d01036c0 0x7f73d0100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d01036c0 0x7f73d0100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52780/0 (socket says 172.21.15.19:52780) 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 -- 172.21.15.19:0/1741073862 learned_addr learned my addr 172.21.15.19:0/1741073862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d569a640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d0102da0 0x7f73d0100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d5e9b640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d010c180 0x7f73d0101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 -- 172.21.15.19:0/1741073862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d010c180 msgr2=0x7f73d0101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d010c180 0x7f73d0101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 -- 172.21.15.19:0/1741073862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d0102da0 msgr2=0x7f73d0100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d0102da0 0x7f73d0100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 -- 172.21.15.19:0/1741073862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d01a6940 con 0x7f73d01036c0 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.997+0000 7f73d4e99640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d01036c0 0x7f73d0100f40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f73c000d950 tx=0x7f73c000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.998+0000 7f73c67fc640 1 -- 172.21.15.19:0/1741073862 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0017070 con 0x7f73d01036c0 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.998+0000 7f73c67fc640 1 -- 172.21.15.19:0/1741073862 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c000de80 con 0x7f73d01036c0 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.998+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d01a6c30 con 0x7f73d01036c0 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.998+0000 7f73c67fc640 1 -- 172.21.15.19:0/1741073862 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c000f620 con 0x7f73d01036c0 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.998+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d01a7170 con 0x7f73d01036c0 2024-09-19T04:05:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:24.999+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d0103180 con 0x7f73d01036c0 2024-09-19T04:05:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.000+0000 7f73c67fc640 1 -- 172.21.15.19:0/1741073862 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f73c0009c30 con 0x7f73d01036c0 2024-09-19T04:05:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.000+0000 7f73c67fc640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f73bc0a3240 0x7f73bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.001+0000 7f73d569a640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f73bc0a3240 0x7f73bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.001+0000 7f73c67fc640 1 -- 172.21.15.19:0/1741073862 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f73c00cf820 con 0x7f73d01036c0 2024-09-19T04:05:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.001+0000 7f73d569a640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f73bc0a3240 0x7f73bc0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009990 tx=0x7f73cc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.005+0000 7f73c67fc640 1 -- 172.21.15.19:0/1741073862 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f73c009c2a0 con 0x7f73d01036c0 2024-09-19T04:05:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.208+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f73d0107270 con 0x7f73bc0a3240 2024-09-19T04:05:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.233+0000 7f73c67fc640 1 -- 172.21.15.19:0/1741073862 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f73cc0095e0 con 0x7f73bc0a3240 2024-09-19T04:05:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f73bc0a3240 msgr2=0x7f73bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f73bc0a3240 0x7f73bc0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009990 tx=0x7f73cc009110 comp rx=0 tx=0).stop 2024-09-19T04:05:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d01036c0 msgr2=0x7f73d0100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d01036c0 0x7f73d0100f40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f73c000d950 tx=0x7f73c000dd50 comp rx=0 tx=0).stop 2024-09-19T04:05:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 shutdown_connections 2024-09-19T04:05:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f73bc0a3240 0x7f73bc0a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d010c180 0x7f73d0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d01036c0 0x7f73d0100f40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 --2- 172.21.15.19:0/1741073862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d0102da0 0x7f73d0100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 >> 172.21.15.19:0/1741073862 conn(0x7f73d00fc7a0 msgr2=0x7f73d00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 shutdown_connections 2024-09-19T04:05:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:25.235+0000 7f73d78fe640 1 -- 172.21.15.19:0/1741073862 wait complete. 2024-09-19T04:05:25.610 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:05:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:05:25.213+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:05:25.611 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:05:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:05:25.213+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:05:25.611 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:05:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:05:25.213+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:05:25.611 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:05:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:05:25.213+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:05:25.611 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:05:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:05:25.213+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:05:25.611 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:05:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: 2024-09-19T04:05:25.178+0000 7f7034aba640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:05:25.611 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:05:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: 2024-09-19T04:05:25.178+0000 7f7034aba640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-19T04:05:25.611 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:05:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: 2024-09-19T04:05:25.178+0000 7f7034aba640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-19T04:05:25.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:25 smithi111 ceph-mon[35365]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T04:05:25.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:25 smithi111 ceph-mon[35365]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:25.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:25 smithi019 ceph-mon[30938]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T04:05:25.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:25 smithi019 ceph-mon[30938]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:25.953 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T04:05:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:25 smithi196 ceph-mon[35731]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T04:05:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:25 smithi196 ceph-mon[35731]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:26.208 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:26.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 -- 172.21.15.19:0/1307823809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 msgr2=0x7f2e1c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 --2- 172.21.15.19:0/1307823809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 0x7f2e1c10ea90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009900 tx=0x7f2e1402a410 comp rx=0 tx=0).stop 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 -- 172.21.15.19:0/1307823809 shutdown_connections 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 --2- 172.21.15.19:0/1307823809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 0x7f2e1c10ea90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 --2- 172.21.15.19:0/1307823809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e1c102790 0x7f2e1c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 --2- 172.21.15.19:0/1307823809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e1c101de0 0x7f2e1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 -- 172.21.15.19:0/1307823809 >> 172.21.15.19:0/1307823809 conn(0x7f2e1c0fd8c0 msgr2=0x7f2e1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 -- 172.21.15.19:0/1307823809 shutdown_connections 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.704+0000 7f2e21727640 1 -- 172.21.15.19:0/1307823809 wait complete. 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e21727640 1 Processor -- start 2024-09-19T04:05:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e21727640 1 -- start start 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e21727640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e1c101de0 0x7f2e1c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e21727640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e1c102790 0x7f2e1c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e21727640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 0x7f2e1c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e21727640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c111f70 con 0x7f2e1c106f10 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e21727640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c111df0 con 0x7f2e1c102790 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e21727640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1120f0 con 0x7f2e1c101de0 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 0x7f2e1c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 0x7f2e1c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52814/0 (socket says 172.21.15.19:52814) 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e1b7fe640 1 -- 172.21.15.19:0/1757018022 learned_addr learned my addr 172.21.15.19:0/1757018022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e1affd640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e1c101de0 0x7f2e1c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e1a7fc640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e1c102790 0x7f2e1c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e1b7fe640 1 -- 172.21.15.19:0/1757018022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e1c101de0 msgr2=0x7f2e1c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:26.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e1b7fe640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e1c101de0 0x7f2e1c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.705+0000 7f2e1b7fe640 1 -- 172.21.15.19:0/1757018022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e1c102790 msgr2=0x7f2e1c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.706+0000 7f2e1b7fe640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e1c102790 0x7f2e1c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.706+0000 7f2e1b7fe640 1 -- 172.21.15.19:0/1757018022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c1a68b0 con 0x7f2e1c106f10 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.706+0000 7f2e1b7fe640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 0x7f2e1c074f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005ab0 tx=0x7f2e140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.706+0000 7f2e03fff640 1 -- 172.21.15.19:0/1757018022 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1403f030 con 0x7f2e1c106f10 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.706+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c1a6b40 con 0x7f2e1c106f10 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.706+0000 7f2e03fff640 1 -- 172.21.15.19:0/1757018022 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e14003ce0 con 0x7f2e1c106f10 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.706+0000 7f2e03fff640 1 -- 172.21.15.19:0/1757018022 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14035070 con 0x7f2e1c106f10 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.706+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c1a7020 con 0x7f2e1c106f10 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.707+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e1c1021c0 con 0x7f2e1c106f10 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.708+0000 7f2e03fff640 1 -- 172.21.15.19:0/1757018022 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2e14042030 con 0x7f2e1c106f10 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.709+0000 7f2e03fff640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2e040a32d0 0x7f2e040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.709+0000 7f2e1affd640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2e040a32d0 0x7f2e040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:26.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.709+0000 7f2e03fff640 1 -- 172.21.15.19:0/1757018022 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2e140f0470 con 0x7f2e1c106f10 2024-09-19T04:05:26.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.709+0000 7f2e1affd640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2e040a32d0 0x7f2e040a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00bbe0 tx=0x7f2e0c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:26.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.713+0000 7f2e03fff640 1 -- 172.21.15.19:0/1757018022 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e140bc740 con 0x7f2e1c106f10 2024-09-19T04:05:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.917+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2e1c062630 con 0x7f2e040a32d0 2024-09-19T04:05:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.943+0000 7f2e03fff640 1 -- 172.21.15.19:0/1757018022 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e1c062630 con 0x7f2e040a32d0 2024-09-19T04:05:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2e040a32d0 msgr2=0x7f2e040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2e040a32d0 0x7f2e040a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00bbe0 tx=0x7f2e0c00b390 comp rx=0 tx=0).stop 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 msgr2=0x7f2e1c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 0x7f2e1c074f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005ab0 tx=0x7f2e140042f0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 shutdown_connections 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2e040a32d0 0x7f2e040a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e1c106f10 0x7f2e1c074f70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e1c102790 0x7f2e1c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 --2- 172.21.15.19:0/1757018022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e1c101de0 0x7f2e1c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 >> 172.21.15.19:0/1757018022 conn(0x7f2e1c0fd8c0 msgr2=0x7f2e1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 shutdown_connections 2024-09-19T04:05:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:26.945+0000 7f2e21727640 1 -- 172.21.15.19:0/1757018022 wait complete. 2024-09-19T04:05:27.602 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T04:05:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:27.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:27 smithi111 ceph-mon[35365]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-19T04:05:27.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:27 smithi111 ceph-mon[35365]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:05:27.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:27 smithi111 ceph-mon[35365]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:27.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:27 smithi019 ceph-mon[30938]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-19T04:05:27.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:27 smithi019 ceph-mon[30938]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:05:27.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:27 smithi019 ceph-mon[30938]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:27.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:27 smithi196 ceph-mon[35731]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-19T04:05:27.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:27 smithi196 ceph-mon[35731]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T04:05:27.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:27 smithi196 ceph-mon[35731]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 -- 172.21.15.19:0/3301728366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904106f10 msgr2=0x7fe90410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/3301728366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904106f10 0x7fe90410ea90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009900 tx=0x7fe8fc02a410 comp rx=0 tx=0).stop 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 -- 172.21.15.19:0/3301728366 shutdown_connections 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/3301728366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904106f10 0x7fe90410ea90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/3301728366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe904102790 0x7fe9041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/3301728366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe904101de0 0x7fe9041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 -- 172.21.15.19:0/3301728366 >> 172.21.15.19:0/3301728366 conn(0x7fe9040fd8c0 msgr2=0x7fe9040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 -- 172.21.15.19:0/3301728366 shutdown_connections 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.336+0000 7fe90aaa2640 1 -- 172.21.15.19:0/3301728366 wait complete. 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90aaa2640 1 Processor -- start 2024-09-19T04:05:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90aaa2640 1 -- start start 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90aaa2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904101de0 0x7fe904198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90aaa2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe904102790 0x7fe9041988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90aaa2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe904106f10 0x7fe90419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90aaa2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe904111f70 con 0x7fe904101de0 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90aaa2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe904111df0 con 0x7fe904102790 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90aaa2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041120f0 con 0x7fe904106f10 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90883e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904101de0 0x7fe904198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90883e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904101de0 0x7fe904198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52834/0 (socket says 172.21.15.19:52834) 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90883e640 1 -- 172.21.15.19:0/372977280 learned_addr learned my addr 172.21.15.19:0/372977280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90903f640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe904106f10 0x7fe90419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe903fff640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe904102790 0x7fe9041988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90883e640 1 -- 172.21.15.19:0/372977280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe904106f10 msgr2=0x7fe90419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90883e640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe904106f10 0x7fe90419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90883e640 1 -- 172.21.15.19:0/372977280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe904102790 msgr2=0x7fe9041988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90883e640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe904102790 0x7fe9041988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.337+0000 7fe90883e640 1 -- 172.21.15.19:0/372977280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90419d270 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.338+0000 7fe90883e640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904101de0 0x7fe904198360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00c920 tx=0x7fe8ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.338+0000 7fe901ffb640 1 -- 172.21.15.19:0/372977280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec007c70 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.338+0000 7fe901ffb640 1 -- 172.21.15.19:0/372977280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8ec007e10 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.338+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90419d560 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.338+0000 7fe901ffb640 1 -- 172.21.15.19:0/372977280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec018890 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.338+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9041a4e40 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.339+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9041021c0 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.340+0000 7fe901ffb640 1 -- 172.21.15.19:0/372977280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe8ec018a70 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.341+0000 7fe901ffb640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8f00a3350 0x7fe8f00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.341+0000 7fe901ffb640 1 -- 172.21.15.19:0/372977280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe8ec0cfe30 con 0x7fe904101de0 2024-09-19T04:05:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.341+0000 7fe903fff640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8f00a3350 0x7fe8f00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:28.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.341+0000 7fe903fff640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8f00a3350 0x7fe8f00a5810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005fd0 tx=0x7fe8f4008230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:28.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.345+0000 7fe901ffb640 1 -- 172.21.15.19:0/372977280 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8ec014030 con 0x7fe904101de0 2024-09-19T04:05:28.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.549+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe9040feae0 con 0x7fe8f00a3350 2024-09-19T04:05:28.555 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/14289782-8419-4ebd-abc8-d18102e04bf9 2024-09-19T04:05:28.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.554+0000 7fe901ffb640 1 -- 172.21.15.19:0/372977280 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe9040feae0 con 0x7fe8f00a3350 2024-09-19T04:05:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8f00a3350 msgr2=0x7fe8f00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8f00a3350 0x7fe8f00a5810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005fd0 tx=0x7fe8f4008230 comp rx=0 tx=0).stop 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904101de0 msgr2=0x7fe904198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904101de0 0x7fe904198360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00c920 tx=0x7fe8ec00cde0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 shutdown_connections 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8f00a3350 0x7fe8f00a5810 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe904106f10 0x7fe90419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe904102790 0x7fe9041988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 --2- 172.21.15.19:0/372977280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe904101de0 0x7fe904198360 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 >> 172.21.15.19:0/372977280 conn(0x7fe9040fd8c0 msgr2=0x7fe9040fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 shutdown_connections 2024-09-19T04:05:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:28.556+0000 7fe90aaa2640 1 -- 172.21.15.19:0/372977280 wait complete. 2024-09-19T04:05:29.268 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T04:05:29.526 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:29.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:29 smithi111 ceph-mon[35365]: pgmap v313: 129 pgs: 129 active+clean; 607 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-19T04:05:29.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:29 smithi111 ceph-mon[35365]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:29.920 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:29 smithi019 ceph-mon[30938]: pgmap v313: 129 pgs: 129 active+clean; 607 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-19T04:05:29.920 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:29 smithi019 ceph-mon[30938]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:29.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:29 smithi196 ceph-mon[35731]: pgmap v313: 129 pgs: 129 active+clean; 607 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-19T04:05:29.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:29 smithi196 ceph-mon[35731]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 -- 172.21.15.19:0/3094859194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0102780 msgr2=0x7f4ac01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/3094859194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0102780 0x7f4ac01067d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009900 tx=0x7f4ab002a410 comp rx=0 tx=0).stop 2024-09-19T04:05:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 -- 172.21.15.19:0/3094859194 shutdown_connections 2024-09-19T04:05:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/3094859194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0106f00 0x7f4ac010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/3094859194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0102780 0x7f4ac01067d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/3094859194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac0101dd0 0x7f4ac01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 -- 172.21.15.19:0/3094859194 >> 172.21.15.19:0/3094859194 conn(0x7f4ac00fd890 msgr2=0x7f4ac00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 -- 172.21.15.19:0/3094859194 shutdown_connections 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.006+0000 7f4ac79d1640 1 -- 172.21.15.19:0/3094859194 wait complete. 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac79d1640 1 Processor -- start 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac79d1640 1 -- start start 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac79d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac0101dd0 0x7f4ac019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac79d1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0102780 0x7f4ac019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac79d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0106f00 0x7f4ac019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac79d1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0111f60 con 0x7f4ac0106f00 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac79d1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0111de0 con 0x7f4ac0102780 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac79d1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01120e0 con 0x7f4ac0101dd0 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac5f6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0106f00 0x7f4ac019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac5f6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0106f00 0x7f4ac019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52854/0 (socket says 172.21.15.19:52854) 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac5f6e640 1 -- 172.21.15.19:0/2451124077 learned_addr learned my addr 172.21.15.19:0/2451124077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac576d640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac0101dd0 0x7f4ac019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac4f6c640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0102780 0x7f4ac019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac5f6e640 1 -- 172.21.15.19:0/2451124077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac0101dd0 msgr2=0x7f4ac019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac5f6e640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac0101dd0 0x7f4ac019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac5f6e640 1 -- 172.21.15.19:0/2451124077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0102780 msgr2=0x7f4ac019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac5f6e640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0102780 0x7f4ac019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac5f6e640 1 -- 172.21.15.19:0/2451124077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac019f620 con 0x7f4ac0106f00 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac576d640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac0101dd0 0x7f4ac019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.007+0000 7f4ac4f6c640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0102780 0x7f4ac019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.008+0000 7f4ac5f6e640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0106f00 0x7f4ac019ef40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800cad0 tx=0x7f4ab800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.008+0000 7f4ab67fc640 1 -- 172.21.15.19:0/2451124077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800ae20 con 0x7f4ac0106f00 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.008+0000 7f4ab67fc640 1 -- 172.21.15.19:0/2451124077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab8005020 con 0x7f4ac0106f00 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.008+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac019f910 con 0x7f4ac0106f00 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.008+0000 7f4ab67fc640 1 -- 172.21.15.19:0/2451124077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800f6f0 con 0x7f4ac0106f00 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.008+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac01a71f0 con 0x7f4ac0106f00 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.009+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac01021b0 con 0x7f4ac0106f00 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.010+0000 7f4ab67fc640 1 -- 172.21.15.19:0/2451124077 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4ab80051d0 con 0x7f4ac0106f00 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.010+0000 7f4ab67fc640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4aac0a3350 0x7f4aac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.011+0000 7f4ac576d640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4aac0a3350 0x7f4aac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.011+0000 7f4ab67fc640 1 -- 172.21.15.19:0/2451124077 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4ab80cfd10 con 0x7f4ac0106f00 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.011+0000 7f4ac576d640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4aac0a3350 0x7f4aac0a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0058c0 tx=0x7f4abc005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.015+0000 7f4ab67fc640 1 -- 172.21.15.19:0/2451124077 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4ab8014030 con 0x7f4ac0106f00 2024-09-19T04:05:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.218+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4ac00fec50 con 0x7f4aac0a3350 2024-09-19T04:05:30.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.244+0000 7f4ab67fc640 1 -- 172.21.15.19:0/2451124077 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ac00fec50 con 0x7f4aac0a3350 2024-09-19T04:05:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4aac0a3350 msgr2=0x7f4aac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4aac0a3350 0x7f4aac0a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0058c0 tx=0x7f4abc005d30 comp rx=0 tx=0).stop 2024-09-19T04:05:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0106f00 msgr2=0x7f4ac019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0106f00 0x7f4ac019ef40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800cad0 tx=0x7f4ab800cf90 comp rx=0 tx=0).stop 2024-09-19T04:05:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 shutdown_connections 2024-09-19T04:05:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4aac0a3350 0x7f4aac0a5810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac0106f00 0x7f4ac019ef40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0102780 0x7f4ac019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 --2- 172.21.15.19:0/2451124077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac0101dd0 0x7f4ac019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 >> 172.21.15.19:0/2451124077 conn(0x7f4ac00fd890 msgr2=0x7f4ac00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 shutdown_connections 2024-09-19T04:05:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:30.246+0000 7f4ac79d1640 1 -- 172.21.15.19:0/2451124077 wait complete. 2024-09-19T04:05:30.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:30 smithi019 ceph-mon[30938]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:30.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:30 smithi111 ceph-mon[35365]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:30.917 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T04:05:30.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:30 smithi196 ceph-mon[35731]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 -- 172.21.15.19:0/2457874582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 msgr2=0x7f16cc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 --2- 172.21.15.19:0/2457874582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 0x7f16cc100ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009930 tx=0x7f16c002a420 comp rx=0 tx=0).stop 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 -- 172.21.15.19:0/2457874582 shutdown_connections 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 --2- 172.21.15.19:0/2457874582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc101420 0x7f16cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 --2- 172.21.15.19:0/2457874582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 0x7f16cc100ee0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 --2- 172.21.15.19:0/2457874582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16cc106a80 0x7f16cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 -- 172.21.15.19:0/2457874582 >> 172.21.15.19:0/2457874582 conn(0x7f16cc0fc7a0 msgr2=0x7f16cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 -- 172.21.15.19:0/2457874582 shutdown_connections 2024-09-19T04:05:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.649+0000 7f16d1012640 1 -- 172.21.15.19:0/2457874582 wait complete. 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.650+0000 7f16d1012640 1 Processor -- start 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.650+0000 7f16d1012640 1 -- start start 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.650+0000 7f16d1012640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 0x7f16cc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.650+0000 7f16d1012640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16cc101420 0x7f16cc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.650+0000 7f16d1012640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc106a80 0x7f16cc1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.650+0000 7f16d1012640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc110ea0 con 0x7f16cc100a80 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16d1012640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc110d20 con 0x7f16cc101420 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16d1012640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc111020 con 0x7f16cc106a80 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 0x7f16cc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 0x7f16cc199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56246/0 (socket says 172.21.15.19:56246) 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 -- 172.21.15.19:0/2438326107 learned_addr learned my addr 172.21.15.19:0/2438326107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16ca59c640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16cc101420 0x7f16cc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cb59e640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc106a80 0x7f16cc1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 -- 172.21.15.19:0/2438326107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc106a80 msgr2=0x7f16cc1a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc106a80 0x7f16cc1a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 -- 172.21.15.19:0/2438326107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16cc101420 msgr2=0x7f16cc199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16cc101420 0x7f16cc199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 -- 172.21.15.19:0/2438326107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc1a48d0 con 0x7f16cc100a80 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cb59e640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc106a80 0x7f16cc1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16cad9d640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 0x7f16cc199310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00b4a0 tx=0x7f16bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.651+0000 7f16b3fff640 1 -- 172.21.15.19:0/2438326107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc014070 con 0x7f16cc100a80 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.652+0000 7f16b3fff640 1 -- 172.21.15.19:0/2438326107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc010af0 con 0x7f16cc100a80 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.652+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc1a4aa0 con 0x7f16cc100a80 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.652+0000 7f16b3fff640 1 -- 172.21.15.19:0/2438326107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc017d10 con 0x7f16cc100a80 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.652+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc1a5000 con 0x7f16cc100a80 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.653+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16cc100ee0 con 0x7f16cc100a80 2024-09-19T04:05:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.654+0000 7f16b3fff640 1 -- 172.21.15.19:0/2438326107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f16bc017070 con 0x7f16cc100a80 2024-09-19T04:05:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.654+0000 7f16b3fff640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16b40a3350 0x7f16b40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.655+0000 7f16ca59c640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16b40a3350 0x7f16b40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.655+0000 7f16b3fff640 1 -- 172.21.15.19:0/2438326107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f16bc0cf200 con 0x7f16cc100a80 2024-09-19T04:05:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.655+0000 7f16ca59c640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16b40a3350 0x7f16b40a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.658+0000 7f16b3fff640 1 -- 172.21.15.19:0/2438326107 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f16bc09bd30 con 0x7f16cc100a80 2024-09-19T04:05:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.862+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f16cc062630 con 0x7f16b40a3350 2024-09-19T04:05:31.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:31 smithi019 ceph-mon[30938]: pgmap v314: 129 pgs: 129 active+clean; 607 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-19T04:05:31.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.866+0000 7f16b3fff640 1 -- 172.21.15.19:0/2438326107 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f16cc062630 con 0x7f16b40a3350 2024-09-19T04:05:31.868 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/ec843e81-1751-4714-98f4-ed467c5abb98 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16b40a3350 msgr2=0x7f16b40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16b40a3350 0x7f16b40a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c00058d0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 msgr2=0x7f16cc199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 0x7f16cc199310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00b4a0 tx=0x7f16bc00b960 comp rx=0 tx=0).stop 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 shutdown_connections 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16b40a3350 0x7f16b40a5810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc106a80 0x7f16cc1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16cc101420 0x7f16cc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 --2- 172.21.15.19:0/2438326107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc100a80 0x7f16cc199310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 >> 172.21.15.19:0/2438326107 conn(0x7f16cc0fc7a0 msgr2=0x7f16cc0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 shutdown_connections 2024-09-19T04:05:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:31.869+0000 7f16d1012640 1 -- 172.21.15.19:0/2438326107 wait complete. 2024-09-19T04:05:31.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:31 smithi111 ceph-mon[35365]: pgmap v314: 129 pgs: 129 active+clean; 607 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-19T04:05:31.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:31 smithi196 ceph-mon[35731]: pgmap v314: 129 pgs: 129 active+clean; 607 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-19T04:05:32.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:05:32.825 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:32.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:32 smithi111 ceph-mon[35365]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:32.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:32 smithi019 ceph-mon[30938]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:32.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:32 smithi196 ceph-mon[35731]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:05:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 -- 172.21.15.19:0/457972267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c102790 msgr2=0x7fc36c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 --2- 172.21.15.19:0/457972267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c102790 0x7fc36c1067e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc358009900 tx=0x7fc35802a410 comp rx=0 tx=0).stop 2024-09-19T04:05:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 -- 172.21.15.19:0/457972267 shutdown_connections 2024-09-19T04:05:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 --2- 172.21.15.19:0/457972267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc36c106f10 0x7fc36c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 --2- 172.21.15.19:0/457972267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c102790 0x7fc36c1067e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 --2- 172.21.15.19:0/457972267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc36c101de0 0x7fc36c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 -- 172.21.15.19:0/457972267 >> 172.21.15.19:0/457972267 conn(0x7fc36c0fd8c0 msgr2=0x7fc36c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 -- 172.21.15.19:0/457972267 shutdown_connections 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.293+0000 7fc373193640 1 -- 172.21.15.19:0/457972267 wait complete. 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc373193640 1 Processor -- start 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc373193640 1 -- start start 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc373193640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc36c101de0 0x7fc36c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc373193640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc36c102790 0x7fc36c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc373193640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c106f10 0x7fc36c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc373193640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c111f70 con 0x7fc36c106f10 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc373193640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c111df0 con 0x7fc36c101de0 2024-09-19T04:05:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc373193640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1120f0 con 0x7fc36c102790 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc371730640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c106f10 0x7fc36c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc371730640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c106f10 0x7fc36c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56258/0 (socket says 172.21.15.19:56258) 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc371730640 1 -- 172.21.15.19:0/1780892837 learned_addr learned my addr 172.21.15.19:0/1780892837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc36bfff640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc36c102790 0x7fc36c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.294+0000 7fc370f2f640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc36c101de0 0x7fc36c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc371730640 1 -- 172.21.15.19:0/1780892837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc36c102790 msgr2=0x7fc36c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc371730640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc36c102790 0x7fc36c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc371730640 1 -- 172.21.15.19:0/1780892837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc36c101de0 msgr2=0x7fc36c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc371730640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc36c101de0 0x7fc36c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc371730640 1 -- 172.21.15.19:0/1780892837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c19f570 con 0x7fc36c106f10 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc370f2f640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc36c101de0 0x7fc36c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc371730640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c106f10 0x7fc36c19ee90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc36400cce0 tx=0x7fc36400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc369ffb640 1 -- 172.21.15.19:0/1780892837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36400f450 con 0x7fc36c106f10 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c19f860 con 0x7fc36c106f10 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc369ffb640 1 -- 172.21.15.19:0/1780892837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc36400fad0 con 0x7fc36c106f10 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc369ffb640 1 -- 172.21.15.19:0/1780892837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364018c10 con 0x7fc36c106f10 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.295+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c1a7140 con 0x7fc36c106f10 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.296+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc36c1021c0 con 0x7fc36c106f10 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.297+0000 7fc369ffb640 1 -- 172.21.15.19:0/1780892837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc36400f5f0 con 0x7fc36c106f10 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.298+0000 7fc369ffb640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc35c0a32d0 0x7fc35c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.298+0000 7fc370f2f640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc35c0a32d0 0x7fc35c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.298+0000 7fc369ffb640 1 -- 172.21.15.19:0/1780892837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc3640d0310 con 0x7fc36c106f10 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.299+0000 7fc370f2f640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc35c0a32d0 0x7fc35c0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc360009b40 tx=0x7fc360005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.302+0000 7fc369ffb640 1 -- 172.21.15.19:0/1780892837 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc364014030 con 0x7fc36c106f10 2024-09-19T04:05:33.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.516+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc36c1a7370 con 0x7fc36c106f10 2024-09-19T04:05:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.516+0000 7fc369ffb640 1 -- 172.21.15.19:0/1780892837 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11980 (secure 0 0 0) 0x7fc36409cc90 con 0x7fc36c106f10 2024-09-19T04:05:33.518 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:33.518 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-19T04:05:23:602377+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:05:23.279052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24454,24452,24448]},"id":1}]} 2024-09-19T04:05:33.518 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-19T04:05:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.518+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc35c0a32d0 msgr2=0x7fc35c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.518+0000 7fc373193640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc35c0a32d0 0x7fc35c0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc360009b40 tx=0x7fc360005e50 comp rx=0 tx=0).stop 2024-09-19T04:05:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.518+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c106f10 msgr2=0x7fc36c19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.518+0000 7fc373193640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c106f10 0x7fc36c19ee90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc36400cce0 tx=0x7fc36400a620 comp rx=0 tx=0).stop 2024-09-19T04:05:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.518+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 shutdown_connections 2024-09-19T04:05:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.519+0000 7fc373193640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc35c0a32d0 0x7fc35c0a5790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.519+0000 7fc373193640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c106f10 0x7fc36c19ee90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.519+0000 7fc373193640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc36c102790 0x7fc36c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.519+0000 7fc373193640 1 --2- 172.21.15.19:0/1780892837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc36c101de0 0x7fc36c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:33.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.519+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 >> 172.21.15.19:0/1780892837 conn(0x7fc36c0fd8c0 msgr2=0x7fc36c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:33.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.519+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 shutdown_connections 2024-09-19T04:05:33.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:33.519+0000 7fc373193640 1 -- 172.21.15.19:0/1780892837 wait complete. 2024-09-19T04:05:33.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:33 smithi019 ceph-mon[30938]: pgmap v315: 129 pgs: 129 active+clean; 607 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-09-19T04:05:33.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:33 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1780892837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:05:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:33 smithi111 ceph-mon[35365]: pgmap v315: 129 pgs: 129 active+clean; 607 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-09-19T04:05:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:33 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1780892837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:05:33.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:33 smithi196 ceph-mon[35731]: pgmap v315: 129 pgs: 129 active+clean; 607 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-09-19T04:05:33.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:33 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1780892837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:05:34.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:05:34.522 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:34.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:34 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:05:34.921 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:34 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:05:34.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:34 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:05:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/1354065897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 msgr2=0x7fd7c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/1354065897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 0x7fd7c410ea90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009ca0 tx=0x7fd7bc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/1354065897 shutdown_connections 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/1354065897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 0x7fd7c410ea90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/1354065897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c4102790 0x7fd7c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/1354065897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c4101de0 0x7fd7c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/1354065897 >> 172.21.15.19:0/1354065897 conn(0x7fd7c40fd8c0 msgr2=0x7fd7c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/1354065897 shutdown_connections 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.002+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/1354065897 wait complete. 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7cc3c3640 1 Processor -- start 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7cc3c3640 1 -- start start 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7cc3c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c4101de0 0x7fd7c41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:35.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7cc3c3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c4102790 0x7fd7c4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7cc3c3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7cc3c3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4111f70 con 0x7fd7c4106f10 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7cc3c3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4111df0 con 0x7fd7c4102790 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7cc3c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c41120f0 con 0x7fd7c4101de0 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7ca960640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7ca960640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56282/0 (socket says 172.21.15.19:56282) 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.003+0000 7fd7ca960640 1 -- 172.21.15.19:0/3908288071 learned_addr learned my addr 172.21.15.19:0/3908288071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7ca15f640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c4101de0 0x7fd7c41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7c995e640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c4102790 0x7fd7c4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7ca960640 1 -- 172.21.15.19:0/3908288071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c4101de0 msgr2=0x7fd7c41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7ca960640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c4101de0 0x7fd7c41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7ca960640 1 -- 172.21.15.19:0/3908288071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c4102790 msgr2=0x7fd7c4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7ca960640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c4102790 0x7fd7c4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7ca960640 1 -- 172.21.15.19:0/3908288071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c419d3a0 con 0x7fd7c4106f10 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7c995e640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c4102790 0x7fd7c4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7ca960640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ccc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc031a20 tx=0x7fd7bc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7bb7fe640 1 -- 172.21.15.19:0/3908288071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc003c30 con 0x7fd7c4106f10 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7bb7fe640 1 -- 172.21.15.19:0/3908288071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7bc003dd0 con 0x7fd7c4106f10 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c419d630 con 0x7fd7c4106f10 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7bb7fe640 1 -- 172.21.15.19:0/3908288071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc02fcf0 con 0x7fd7c4106f10 2024-09-19T04:05:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.004+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c41a4f70 con 0x7fd7c4106f10 2024-09-19T04:05:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.005+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c41021c0 con 0x7fd7c4106f10 2024-09-19T04:05:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.006+0000 7fd7bb7fe640 1 -- 172.21.15.19:0/3908288071 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd7bc004050 con 0x7fd7c4106f10 2024-09-19T04:05:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.007+0000 7fd7bb7fe640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7b00a32d0 0x7fd7b00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.007+0000 7fd7ca15f640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7b00a32d0 0x7fd7b00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.007+0000 7fd7bb7fe640 1 -- 172.21.15.19:0/3908288071 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd7bc0f1160 con 0x7fd7c4106f10 2024-09-19T04:05:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.008+0000 7fd7ca15f640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7b00a32d0 0x7fd7b00a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000bbe0 tx=0x7fd7c000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.010+0000 7fd7bb7fe640 1 -- 172.21.15.19:0/3908288071 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd7bc0bdc20 con 0x7fd7c4106f10 2024-09-19T04:05:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.193+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7c4199750 con 0x7fd7c4106f10 2024-09-19T04:05:35.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.195+0000 7fd7bb7fe640 1 -- 172.21.15.19:0/3908288071 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24188 (secure 0 0 0) 0x7fd7c4199750 con 0x7fd7c4106f10 2024-09-19T04:05:35.196 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:04:48.953123+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:05:35.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7b00a32d0 msgr2=0x7fd7b00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:35.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7b00a32d0 0x7fd7b00a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000bbe0 tx=0x7fd7c000b390 comp rx=0 tx=0).stop 2024-09-19T04:05:35.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 msgr2=0x7fd7c419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:35.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ccc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc031a20 tx=0x7fd7bc031a50 comp rx=0 tx=0).stop 2024-09-19T04:05:35.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 shutdown_connections 2024-09-19T04:05:35.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7b00a32d0 0x7fd7b00a5790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ccc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c4102790 0x7fd7c4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 --2- 172.21.15.19:0/3908288071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c4101de0 0x7fd7c41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:35.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 >> 172.21.15.19:0/3908288071 conn(0x7fd7c40fd8c0 msgr2=0x7fd7c40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:35.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 shutdown_connections 2024-09-19T04:05:35.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:35.197+0000 7fd7cc3c3640 1 -- 172.21.15.19:0/3908288071 wait complete. 2024-09-19T04:05:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:35 smithi111 ceph-mon[35365]: pgmap v316: 129 pgs: 129 active+clean; 626 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 22 op/s 2024-09-19T04:05:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:35 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3908288071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:35.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:35 smithi019 ceph-mon[30938]: pgmap v316: 129 pgs: 129 active+clean; 626 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 22 op/s 2024-09-19T04:05:35.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:35 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3908288071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:35.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:35 smithi196 ceph-mon[35731]: pgmap v316: 129 pgs: 129 active+clean; 626 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 22 op/s 2024-09-19T04:05:35.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:35 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3908288071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:36.068 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:05:36.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:05:36.568 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:36.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:36 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:36.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:36 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:36.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:36 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:36.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:36 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.071+0000 7f6f25b52640 1 -- 172.21.15.19:0/1407941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20101de0 msgr2=0x7f6f201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.071+0000 7f6f25b52640 1 --2- 172.21.15.19:0/1407941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20101de0 0x7f6f201021c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009900 tx=0x7f6f1002a410 comp rx=0 tx=0).stop 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 -- 172.21.15.19:0/1407941 shutdown_connections 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 --2- 172.21.15.19:0/1407941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f20106f10 0x7f6f2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 --2- 172.21.15.19:0/1407941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f20102790 0x7f6f201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 --2- 172.21.15.19:0/1407941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20101de0 0x7f6f201021c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 -- 172.21.15.19:0/1407941 >> 172.21.15.19:0/1407941 conn(0x7f6f200fd8c0 msgr2=0x7f6f200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 -- 172.21.15.19:0/1407941 shutdown_connections 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 -- 172.21.15.19:0/1407941 wait complete. 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 Processor -- start 2024-09-19T04:05:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.072+0000 7f6f25b52640 1 -- start start 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f25b52640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f20101de0 0x7f6f20198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f25b52640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f20102790 0x7f6f20198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f25b52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20106f10 0x7f6f2019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f25b52640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20112010 con 0x7f6f20106f10 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f25b52640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20111e90 con 0x7f6f20101de0 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f25b52640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20112190 con 0x7f6f20102790 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20106f10 0x7f6f2019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20106f10 0x7f6f2019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56300/0 (socket says 172.21.15.19:56300) 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 -- 172.21.15.19:0/2484388140 learned_addr learned my addr 172.21.15.19:0/2484388140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1f7fe640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f20101de0 0x7f6f20198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1effd640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f20102790 0x7f6f20198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 -- 172.21.15.19:0/2484388140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f20102790 msgr2=0x7f6f20198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f20102790 0x7f6f20198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 -- 172.21.15.19:0/2484388140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f20101de0 msgr2=0x7f6f20198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f20101de0 0x7f6f20198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 -- 172.21.15.19:0/2484388140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2019d5c0 con 0x7f6f20106f10 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1ffff640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20106f10 0x7f6f2019cee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800b970 tx=0x7f6f1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.073+0000 7f6f1cff9640 1 -- 172.21.15.19:0/2484388140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18019070 con 0x7f6f20106f10 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.074+0000 7f6f1cff9640 1 -- 172.21.15.19:0/2484388140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f1800cb40 con 0x7f6f20106f10 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.074+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2019d8b0 con 0x7f6f20106f10 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.074+0000 7f6f1cff9640 1 -- 172.21.15.19:0/2484388140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18007650 con 0x7f6f20106f10 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.074+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f201a5190 con 0x7f6f20106f10 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.075+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f20102240 con 0x7f6f20106f10 2024-09-19T04:05:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.076+0000 7f6f1cff9640 1 -- 172.21.15.19:0/2484388140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6f1801c070 con 0x7f6f20106f10 2024-09-19T04:05:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.076+0000 7f6f1cff9640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6f080a32d0 0x7f6f080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.077+0000 7f6f1f7fe640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6f080a32d0 0x7f6f080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.077+0000 7f6f1cff9640 1 -- 172.21.15.19:0/2484388140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6f180cf220 con 0x7f6f20106f10 2024-09-19T04:05:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.077+0000 7f6f1f7fe640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6f080a32d0 0x7f6f080a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f10000c00 tx=0x7f6f100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.080+0000 7f6f1cff9640 1 -- 172.21.15.19:0/2484388140 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6f1809b600 con 0x7f6f20106f10 2024-09-19T04:05:37.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:36 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:36 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:36 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:36 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:36 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:36 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:36 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:36 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:37.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.265+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f20062630 con 0x7f6f080a32d0 2024-09-19T04:05:37.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.281+0000 7f6f1cff9640 1 -- 172.21.15.19:0/2484388140 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340641 (secure 0 0 0) 0x7f6f20062630 con 0x7f6f080a32d0 2024-09-19T04:05:37.286 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:37.288 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-19T04:05:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6f080a32d0 msgr2=0x7f6f080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6f080a32d0 0x7f6f080a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f10000c00 tx=0x7f6f100058d0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20106f10 msgr2=0x7f6f2019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20106f10 0x7f6f2019cee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800b970 tx=0x7f6f1800be30 comp rx=0 tx=0).stop 2024-09-19T04:05:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 shutdown_connections 2024-09-19T04:05:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6f080a32d0 0x7f6f080a5790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f20106f10 0x7f6f2019cee0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f20102790 0x7f6f20198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 --2- 172.21.15.19:0/2484388140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f20101de0 0x7f6f20198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:37.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 >> 172.21.15.19:0/2484388140 conn(0x7f6f200fd8c0 msgr2=0x7f6f200feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:37.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 shutdown_connections 2024-09-19T04:05:37.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:37.285+0000 7f6f25b52640 1 -- 172.21.15.19:0/2484388140 wait complete. 2024-09-19T04:05:38.016 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-19T04:05:36.422041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640762,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":281,"num_read_kb":243,"num_write":414,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348776,"kb_used_data":12008,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593048,"statfs":{"total":1151940427776,"available":1151583281152,"internally_reserved":0,"allocated":12296192,"data_stored":7786712,"data_compressed":317721,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42760,"num_objects":27,"num_object_clones":0,"num_object_copies":83,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":69,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003405"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:31.621183+0000","last_change":"2024-09-19T04:04:37.214769+0000","last_active":"2024-09-19T04:05:31.621183+0000","last_peered":"2024-09-19T04:05:31.621183+0000","last_clean":"2024-09-19T04:05:31.621183+0000","last_became_active":"2024-09-19T04:04:37.214518+0000","last_became_peered":"2024-09-19T04:04:37.214518+0000","last_unstale":"2024-09-19T04:05:31.621183+0000","last_undegraded":"2024-09-19T04:05:31.621183+0000","last_fullsized":"2024-09-19T04:05:31.621183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:50.704566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885289+0000","last_change":"2024-09-19T04:04:39.329733+0000","last_active":"2024-09-19T04:04:49.885289+0000","last_peered":"2024-09-19T04:04:49.885289+0000","last_clean":"2024-09-19T04:04:49.885289+0000","last_became_active":"2024-09-19T04:04:39.329031+0000","last_became_peered":"2024-09-19T04:04:39.329031+0000","last_unstale":"2024-09-19T04:04:49.885289+0000","last_undegraded":"2024-09-19T04:04:49.885289+0000","last_fullsized":"2024-09-19T04:04:49.885289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:57.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351473+0000","last_change":"2024-09-19T04:04:46.362679+0000","last_active":"2024-09-19T04:04:49.351473+0000","last_peered":"2024-09-19T04:04:49.351473+0000","last_clean":"2024-09-19T04:04:49.351473+0000","last_became_active":"2024-09-19T04:04:46.362257+0000","last_became_peered":"2024-09-19T04:04:46.362257+0000","last_unstale":"2024-09-19T04:04:49.351473+0000","last_undegraded":"2024-09-19T04:04:49.351473+0000","last_fullsized":"2024-09-19T04:04:49.351473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.389839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885530+0000","last_change":"2024-09-19T04:04:37.647295+0000","last_active":"2024-09-19T04:04:49.885530+0000","last_peered":"2024-09-19T04:04:49.885530+0000","last_clean":"2024-09-19T04:04:49.885530+0000","last_became_active":"2024-09-19T04:04:37.646964+0000","last_became_peered":"2024-09-19T04:04:37.646964+0000","last_unstale":"2024-09-19T04:04:49.885530+0000","last_undegraded":"2024-09-19T04:04:49.885530+0000","last_fullsized":"2024-09-19T04:04:49.885530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:49.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905863+0000","last_change":"2024-09-19T04:04:39.318270+0000","last_active":"2024-09-19T04:04:49.905863+0000","last_peered":"2024-09-19T04:04:49.905863+0000","last_clean":"2024-09-19T04:04:49.905863+0000","last_became_active":"2024-09-19T04:04:39.317972+0000","last_became_peered":"2024-09-19T04:04:39.317972+0000","last_unstale":"2024-09-19T04:04:49.905863+0000","last_undegraded":"2024-09-19T04:04:49.905863+0000","last_fullsized":"2024-09-19T04:04:49.905863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:25.994112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904693+0000","last_change":"2024-09-19T04:04:46.355111+0000","last_active":"2024-09-19T04:04:49.904693+0000","last_peered":"2024-09-19T04:04:49.904693+0000","last_clean":"2024-09-19T04:04:49.904693+0000","last_became_active":"2024-09-19T04:04:46.354684+0000","last_became_peered":"2024-09-19T04:04:46.354684+0000","last_unstale":"2024-09-19T04:04:49.904693+0000","last_undegraded":"2024-09-19T04:04:49.904693+0000","last_fullsized":"2024-09-19T04:04:49.904693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:33.141690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.606646+0000","last_change":"2024-09-19T04:04:37.211772+0000","last_active":"2024-09-19T04:05:22.606646+0000","last_peered":"2024-09-19T04:05:22.606646+0000","last_clean":"2024-09-19T04:05:22.606646+0000","last_became_active":"2024-09-19T04:04:37.211452+0000","last_became_peered":"2024-09-19T04:04:37.211452+0000","last_unstale":"2024-09-19T04:05:22.606646+0000","last_undegraded":"2024-09-19T04:05:22.606646+0000","last_fullsized":"2024-09-19T04:05:22.606646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:57.781481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905675+0000","last_change":"2024-09-19T04:04:39.320791+0000","last_active":"2024-09-19T04:04:49.905675+0000","last_peered":"2024-09-19T04:04:49.905675+0000","last_clean":"2024-09-19T04:04:49.905675+0000","last_became_active":"2024-09-19T04:04:39.320206+0000","last_became_peered":"2024-09-19T04:04:39.320206+0000","last_unstale":"2024-09-19T04:04:49.905675+0000","last_undegraded":"2024-09-19T04:04:49.905675+0000","last_fullsized":"2024-09-19T04:04:49.905675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:17.385974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905406+0000","last_change":"2024-09-19T04:04:46.363375+0000","last_active":"2024-09-19T04:04:49.905406+0000","last_peered":"2024-09-19T04:04:49.905406+0000","last_clean":"2024-09-19T04:04:49.905406+0000","last_became_active":"2024-09-19T04:04:46.362800+0000","last_became_peered":"2024-09-19T04:04:46.362800+0000","last_unstale":"2024-09-19T04:04:49.905406+0000","last_undegraded":"2024-09-19T04:04:49.905406+0000","last_fullsized":"2024-09-19T04:04:49.905406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:55.651507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529934+0000","last_change":"2024-09-19T04:04:37.206218+0000","last_active":"2024-09-19T04:05:22.529934+0000","last_peered":"2024-09-19T04:05:22.529934+0000","last_clean":"2024-09-19T04:05:22.529934+0000","last_became_active":"2024-09-19T04:04:37.205675+0000","last_became_peered":"2024-09-19T04:04:37.205675+0000","last_unstale":"2024-09-19T04:05:22.529934+0000","last_undegraded":"2024-09-19T04:05:22.529934+0000","last_fullsized":"2024-09-19T04:05:22.529934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:27.664960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906012+0000","last_change":"2024-09-19T04:04:39.329667+0000","last_active":"2024-09-19T04:04:49.906012+0000","last_peered":"2024-09-19T04:04:49.906012+0000","last_clean":"2024-09-19T04:04:49.906012+0000","last_became_active":"2024-09-19T04:04:39.328180+0000","last_became_peered":"2024-09-19T04:04:39.328180+0000","last_unstale":"2024-09-19T04:04:49.906012+0000","last_undegraded":"2024-09-19T04:04:49.906012+0000","last_fullsized":"2024-09-19T04:04:49.906012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:59.782472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905190+0000","last_change":"2024-09-19T04:04:46.359432+0000","last_active":"2024-09-19T04:04:49.905190+0000","last_peered":"2024-09-19T04:04:49.905190+0000","last_clean":"2024-09-19T04:04:49.905190+0000","last_became_active":"2024-09-19T04:04:46.358966+0000","last_became_peered":"2024-09-19T04:04:46.358966+0000","last_unstale":"2024-09-19T04:04:49.905190+0000","last_undegraded":"2024-09-19T04:04:49.905190+0000","last_fullsized":"2024-09-19T04:04:49.905190+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:41.537926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905315+0000","last_change":"2024-09-19T04:04:39.325084+0000","last_active":"2024-09-19T04:04:49.905315+0000","last_peered":"2024-09-19T04:04:49.905315+0000","last_clean":"2024-09-19T04:04:49.905315+0000","last_became_active":"2024-09-19T04:04:39.324126+0000","last_became_peered":"2024-09-19T04:04:39.324126+0000","last_unstale":"2024-09-19T04:04:49.905315+0000","last_undegraded":"2024-09-19T04:04:49.905315+0000","last_fullsized":"2024-09-19T04:04:49.905315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:13.523889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:19.503198+0000","last_change":"2024-09-19T04:04:37.211548+0000","last_active":"2024-09-19T04:05:19.503198+0000","last_peered":"2024-09-19T04:05:19.503198+0000","last_clean":"2024-09-19T04:05:19.503198+0000","last_became_active":"2024-09-19T04:04:37.211232+0000","last_became_peered":"2024-09-19T04:04:37.211232+0000","last_unstale":"2024-09-19T04:05:19.503198+0000","last_undegraded":"2024-09-19T04:05:19.503198+0000","last_fullsized":"2024-09-19T04:05:19.503198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:35:06.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905215+0000","last_change":"2024-09-19T04:04:46.374921+0000","last_active":"2024-09-19T04:04:49.905215+0000","last_peered":"2024-09-19T04:04:49.905215+0000","last_clean":"2024-09-19T04:04:49.905215+0000","last_became_active":"2024-09-19T04:04:46.374466+0000","last_became_peered":"2024-09-19T04:04:46.374466+0000","last_unstale":"2024-09-19T04:04:49.905215+0000","last_undegraded":"2024-09-19T04:04:49.905215+0000","last_fullsized":"2024-09-19T04:04:49.905215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:46.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884937+0000","last_change":"2024-09-19T04:04:39.319948+0000","last_active":"2024-09-19T04:04:49.884937+0000","last_peered":"2024-09-19T04:04:49.884937+0000","last_clean":"2024-09-19T04:04:49.884937+0000","last_became_active":"2024-09-19T04:04:39.319675+0000","last_became_peered":"2024-09-19T04:04:39.319675+0000","last_unstale":"2024-09-19T04:04:49.884937+0000","last_undegraded":"2024-09-19T04:04:49.884937+0000","last_fullsized":"2024-09-19T04:04:49.884937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:08.818133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:21.514500+0000","last_change":"2024-09-19T04:04:37.207705+0000","last_active":"2024-09-19T04:05:21.514500+0000","last_peered":"2024-09-19T04:05:21.514500+0000","last_clean":"2024-09-19T04:05:21.514500+0000","last_became_active":"2024-09-19T04:04:37.207251+0000","last_became_peered":"2024-09-19T04:04:37.207251+0000","last_unstale":"2024-09-19T04:05:21.514500+0000","last_undegraded":"2024-09-19T04:05:21.514500+0000","last_fullsized":"2024-09-19T04:05:21.514500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:20.835004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:50.337968+0000","last_change":"2024-09-19T04:04:46.354463+0000","last_active":"2024-09-19T04:04:50.337968+0000","last_peered":"2024-09-19T04:04:50.337968+0000","last_clean":"2024-09-19T04:04:50.337968+0000","last_became_active":"2024-09-19T04:04:46.353754+0000","last_became_peered":"2024-09-19T04:04:46.353754+0000","last_unstale":"2024-09-19T04:04:50.337968+0000","last_undegraded":"2024-09-19T04:04:50.337968+0000","last_fullsized":"2024-09-19T04:04:50.337968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:09.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904504+0000","last_change":"2024-09-19T04:04:39.322727+0000","last_active":"2024-09-19T04:04:49.904504+0000","last_peered":"2024-09-19T04:04:49.904504+0000","last_clean":"2024-09-19T04:04:49.904504+0000","last_became_active":"2024-09-19T04:04:39.320989+0000","last_became_peered":"2024-09-19T04:04:39.320989+0000","last_unstale":"2024-09-19T04:04:49.904504+0000","last_undegraded":"2024-09-19T04:04:49.904504+0000","last_fullsized":"2024-09-19T04:04:49.904504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:06.536540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529961+0000","last_change":"2024-09-19T04:04:37.213602+0000","last_active":"2024-09-19T04:05:22.529961+0000","last_peered":"2024-09-19T04:05:22.529961+0000","last_clean":"2024-09-19T04:05:22.529961+0000","last_became_active":"2024-09-19T04:04:37.213240+0000","last_became_peered":"2024-09-19T04:04:37.213240+0000","last_unstale":"2024-09-19T04:05:22.529961+0000","last_undegraded":"2024-09-19T04:05:22.529961+0000","last_fullsized":"2024-09-19T04:05:22.529961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:09.822393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350925+0000","last_change":"2024-09-19T04:04:46.363921+0000","last_active":"2024-09-19T04:04:49.350925+0000","last_peered":"2024-09-19T04:04:49.350925+0000","last_clean":"2024-09-19T04:04:49.350925+0000","last_became_active":"2024-09-19T04:04:46.363476+0000","last_became_peered":"2024-09-19T04:04:46.363476+0000","last_unstale":"2024-09-19T04:04:49.350925+0000","last_undegraded":"2024-09-19T04:04:49.350925+0000","last_fullsized":"2024-09-19T04:04:49.350925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:32.015782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904206+0000","last_change":"2024-09-19T04:04:39.316814+0000","last_active":"2024-09-19T04:04:49.904206+0000","last_peered":"2024-09-19T04:04:49.904206+0000","last_clean":"2024-09-19T04:04:49.904206+0000","last_became_active":"2024-09-19T04:04:39.316504+0000","last_became_peered":"2024-09-19T04:04:39.316504+0000","last_unstale":"2024-09-19T04:04:49.904206+0000","last_undegraded":"2024-09-19T04:04:49.904206+0000","last_fullsized":"2024-09-19T04:04:49.904206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:19.503333+0000","last_change":"2024-09-19T04:04:37.647260+0000","last_active":"2024-09-19T04:05:19.503333+0000","last_peered":"2024-09-19T04:05:19.503333+0000","last_clean":"2024-09-19T04:05:19.503333+0000","last_became_active":"2024-09-19T04:04:37.646971+0000","last_became_peered":"2024-09-19T04:04:37.646971+0000","last_unstale":"2024-09-19T04:05:19.503333+0000","last_undegraded":"2024-09-19T04:05:19.503333+0000","last_fullsized":"2024-09-19T04:05:19.503333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:07.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905176+0000","last_change":"2024-09-19T04:04:46.375999+0000","last_active":"2024-09-19T04:04:49.905176+0000","last_peered":"2024-09-19T04:04:49.905176+0000","last_clean":"2024-09-19T04:04:49.905176+0000","last_became_active":"2024-09-19T04:04:46.375602+0000","last_became_peered":"2024-09-19T04:04:46.375602+0000","last_unstale":"2024-09-19T04:04:49.905176+0000","last_undegraded":"2024-09-19T04:04:49.905176+0000","last_fullsized":"2024-09-19T04:04:49.905176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:18.952629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905926+0000","last_change":"2024-09-19T04:04:39.318185+0000","last_active":"2024-09-19T04:04:49.905926+0000","last_peered":"2024-09-19T04:04:49.905926+0000","last_clean":"2024-09-19T04:04:49.905926+0000","last_became_active":"2024-09-19T04:04:39.317885+0000","last_became_peered":"2024-09-19T04:04:39.317885+0000","last_unstale":"2024-09-19T04:04:49.905926+0000","last_undegraded":"2024-09-19T04:04:49.905926+0000","last_fullsized":"2024-09-19T04:04:49.905926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:27.719248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529369+0000","last_change":"2024-09-19T04:04:37.703740+0000","last_active":"2024-09-19T04:05:22.529369+0000","last_peered":"2024-09-19T04:05:22.529369+0000","last_clean":"2024-09-19T04:05:22.529369+0000","last_became_active":"2024-09-19T04:04:37.703414+0000","last_became_peered":"2024-09-19T04:04:37.703414+0000","last_unstale":"2024-09-19T04:05:22.529369+0000","last_undegraded":"2024-09-19T04:05:22.529369+0000","last_fullsized":"2024-09-19T04:05:22.529369+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:19.653559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351586+0000","last_change":"2024-09-19T04:04:46.354964+0000","last_active":"2024-09-19T04:04:49.351586+0000","last_peered":"2024-09-19T04:04:49.351586+0000","last_clean":"2024-09-19T04:04:49.351586+0000","last_became_active":"2024-09-19T04:04:46.354494+0000","last_became_peered":"2024-09-19T04:04:46.354494+0000","last_unstale":"2024-09-19T04:04:49.351586+0000","last_undegraded":"2024-09-19T04:04:49.351586+0000","last_fullsized":"2024-09-19T04:04:49.351586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:11.246134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885406+0000","last_change":"2024-09-19T04:04:39.315811+0000","last_active":"2024-09-19T04:04:49.885406+0000","last_peered":"2024-09-19T04:04:49.885406+0000","last_clean":"2024-09-19T04:04:49.885406+0000","last_became_active":"2024-09-19T04:04:39.315466+0000","last_became_peered":"2024-09-19T04:04:39.315466+0000","last_unstale":"2024-09-19T04:04:49.885406+0000","last_undegraded":"2024-09-19T04:04:49.885406+0000","last_fullsized":"2024-09-19T04:04:49.885406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:17.414765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:30.316843+0000","last_change":"2024-09-19T04:04:37.204206+0000","last_active":"2024-09-19T04:05:30.316843+0000","last_peered":"2024-09-19T04:05:30.316843+0000","last_clean":"2024-09-19T04:05:30.316843+0000","last_became_active":"2024-09-19T04:04:37.203845+0000","last_became_peered":"2024-09-19T04:04:37.203845+0000","last_unstale":"2024-09-19T04:05:30.316843+0000","last_undegraded":"2024-09-19T04:05:30.316843+0000","last_fullsized":"2024-09-19T04:05:30.316843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:52.868643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884707+0000","last_change":"2024-09-19T04:04:46.375541+0000","last_active":"2024-09-19T04:04:49.884707+0000","last_peered":"2024-09-19T04:04:49.884707+0000","last_clean":"2024-09-19T04:04:49.884707+0000","last_became_active":"2024-09-19T04:04:46.375142+0000","last_became_peered":"2024-09-19T04:04:46.375142+0000","last_unstale":"2024-09-19T04:04:49.884707+0000","last_undegraded":"2024-09-19T04:04:49.884707+0000","last_fullsized":"2024-09-19T04:04:49.884707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.889459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904952+0000","last_change":"2024-09-19T04:04:39.324094+0000","last_active":"2024-09-19T04:04:49.904952+0000","last_peered":"2024-09-19T04:04:49.904952+0000","last_clean":"2024-09-19T04:04:49.904952+0000","last_became_active":"2024-09-19T04:04:39.322897+0000","last_became_peered":"2024-09-19T04:04:39.322897+0000","last_unstale":"2024-09-19T04:04:49.904952+0000","last_undegraded":"2024-09-19T04:04:49.904952+0000","last_fullsized":"2024-09-19T04:04:49.904952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:07.773110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529315+0000","last_change":"2024-09-19T04:04:37.647686+0000","last_active":"2024-09-19T04:05:22.529315+0000","last_peered":"2024-09-19T04:05:22.529315+0000","last_clean":"2024-09-19T04:05:22.529315+0000","last_became_active":"2024-09-19T04:04:37.646979+0000","last_became_peered":"2024-09-19T04:04:37.646979+0000","last_unstale":"2024-09-19T04:05:22.529315+0000","last_undegraded":"2024-09-19T04:05:22.529315+0000","last_fullsized":"2024-09-19T04:05:22.529315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:48.063075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905896+0000","last_change":"2024-09-19T04:04:46.356078+0000","last_active":"2024-09-19T04:04:49.905896+0000","last_peered":"2024-09-19T04:04:49.905896+0000","last_clean":"2024-09-19T04:04:49.905896+0000","last_became_active":"2024-09-19T04:04:46.355630+0000","last_became_peered":"2024-09-19T04:04:46.355630+0000","last_unstale":"2024-09-19T04:04:49.905896+0000","last_undegraded":"2024-09-19T04:04:49.905896+0000","last_fullsized":"2024-09-19T04:04:49.905896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:45.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351620+0000","last_change":"2024-09-19T04:04:39.311067+0000","last_active":"2024-09-19T04:04:49.351620+0000","last_peered":"2024-09-19T04:04:49.351620+0000","last_clean":"2024-09-19T04:04:49.351620+0000","last_became_active":"2024-09-19T04:04:39.310725+0000","last_became_peered":"2024-09-19T04:04:39.310725+0000","last_unstale":"2024-09-19T04:04:49.351620+0000","last_undegraded":"2024-09-19T04:04:49.351620+0000","last_fullsized":"2024-09-19T04:04:49.351620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:47:11.488121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351028+0000","last_change":"2024-09-19T04:04:37.648335+0000","last_active":"2024-09-19T04:04:49.351028+0000","last_peered":"2024-09-19T04:04:49.351028+0000","last_clean":"2024-09-19T04:04:49.351028+0000","last_became_active":"2024-09-19T04:04:37.648000+0000","last_became_peered":"2024-09-19T04:04:37.648000+0000","last_unstale":"2024-09-19T04:04:49.351028+0000","last_undegraded":"2024-09-19T04:04:49.351028+0000","last_fullsized":"2024-09-19T04:04:49.351028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:16.102728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904470+0000","last_change":"2024-09-19T04:04:46.375653+0000","last_active":"2024-09-19T04:04:49.904470+0000","last_peered":"2024-09-19T04:04:49.904470+0000","last_clean":"2024-09-19T04:04:49.904470+0000","last_became_active":"2024-09-19T04:04:46.375132+0000","last_became_peered":"2024-09-19T04:04:46.375132+0000","last_unstale":"2024-09-19T04:04:49.904470+0000","last_undegraded":"2024-09-19T04:04:49.904470+0000","last_fullsized":"2024-09-19T04:04:49.904470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:03.332697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351643+0000","last_change":"2024-09-19T04:04:39.318155+0000","last_active":"2024-09-19T04:04:49.351643+0000","last_peered":"2024-09-19T04:04:49.351643+0000","last_clean":"2024-09-19T04:04:49.351643+0000","last_became_active":"2024-09-19T04:04:39.316710+0000","last_became_peered":"2024-09-19T04:04:39.316710+0000","last_unstale":"2024-09-19T04:04:49.351643+0000","last_undegraded":"2024-09-19T04:04:49.351643+0000","last_fullsized":"2024-09-19T04:04:49.351643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:59.084523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:33.624368+0000","last_change":"2024-09-19T04:04:37.210695+0000","last_active":"2024-09-19T04:05:33.624368+0000","last_peered":"2024-09-19T04:05:33.624368+0000","last_clean":"2024-09-19T04:05:33.624368+0000","last_became_active":"2024-09-19T04:04:37.210334+0000","last_became_peered":"2024-09-19T04:04:37.210334+0000","last_unstale":"2024-09-19T04:05:33.624368+0000","last_undegraded":"2024-09-19T04:05:33.624368+0000","last_fullsized":"2024-09-19T04:05:33.624368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:12.545248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904818+0000","last_change":"2024-09-19T04:04:46.358130+0000","last_active":"2024-09-19T04:04:49.904818+0000","last_peered":"2024-09-19T04:04:49.904818+0000","last_clean":"2024-09-19T04:04:49.904818+0000","last_became_active":"2024-09-19T04:04:46.356810+0000","last_became_peered":"2024-09-19T04:04:46.356810+0000","last_unstale":"2024-09-19T04:04:49.904818+0000","last_undegraded":"2024-09-19T04:04:49.904818+0000","last_fullsized":"2024-09-19T04:04:49.904818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:49.884478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905394+0000","last_change":"2024-09-19T04:04:39.325866+0000","last_active":"2024-09-19T04:04:49.905394+0000","last_peered":"2024-09-19T04:04:49.905394+0000","last_clean":"2024-09-19T04:04:49.905394+0000","last_became_active":"2024-09-19T04:04:39.325473+0000","last_became_peered":"2024-09-19T04:04:39.325473+0000","last_unstale":"2024-09-19T04:04:49.905394+0000","last_undegraded":"2024-09-19T04:04:49.905394+0000","last_fullsized":"2024-09-19T04:04:49.905394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:30.036704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:34.625947+0000","last_change":"2024-09-19T04:04:37.647310+0000","last_active":"2024-09-19T04:05:34.625947+0000","last_peered":"2024-09-19T04:05:34.625947+0000","last_clean":"2024-09-19T04:05:34.625947+0000","last_became_active":"2024-09-19T04:04:37.646903+0000","last_became_peered":"2024-09-19T04:04:37.646903+0000","last_unstale":"2024-09-19T04:05:34.625947+0000","last_undegraded":"2024-09-19T04:05:34.625947+0000","last_fullsized":"2024-09-19T04:05:34.625947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:20.097472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904925+0000","last_change":"2024-09-19T04:04:46.375261+0000","last_active":"2024-09-19T04:04:49.904925+0000","last_peered":"2024-09-19T04:04:49.904925+0000","last_clean":"2024-09-19T04:04:49.904925+0000","last_became_active":"2024-09-19T04:04:46.374489+0000","last_became_peered":"2024-09-19T04:04:46.374489+0000","last_unstale":"2024-09-19T04:04:49.904925+0000","last_undegraded":"2024-09-19T04:04:49.904925+0000","last_fullsized":"2024-09-19T04:04:49.904925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:05.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904224+0000","last_change":"2024-09-19T04:04:39.322265+0000","last_active":"2024-09-19T04:04:49.904224+0000","last_peered":"2024-09-19T04:04:49.904224+0000","last_clean":"2024-09-19T04:04:49.904224+0000","last_became_active":"2024-09-19T04:04:39.321938+0000","last_became_peered":"2024-09-19T04:04:39.321938+0000","last_unstale":"2024-09-19T04:04:49.904224+0000","last_undegraded":"2024-09-19T04:04:49.904224+0000","last_fullsized":"2024-09-19T04:04:49.904224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:19.220661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:20.508304+0000","last_change":"2024-09-19T04:04:37.211330+0000","last_active":"2024-09-19T04:05:20.508304+0000","last_peered":"2024-09-19T04:05:20.508304+0000","last_clean":"2024-09-19T04:05:20.508304+0000","last_became_active":"2024-09-19T04:04:37.210788+0000","last_became_peered":"2024-09-19T04:04:37.210788+0000","last_unstale":"2024-09-19T04:05:20.508304+0000","last_undegraded":"2024-09-19T04:05:20.508304+0000","last_fullsized":"2024-09-19T04:05:20.508304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:43.169897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905589+0000","last_change":"2024-09-19T04:04:46.376030+0000","last_active":"2024-09-19T04:04:49.905589+0000","last_peered":"2024-09-19T04:04:49.905589+0000","last_clean":"2024-09-19T04:04:49.905589+0000","last_became_active":"2024-09-19T04:04:46.375386+0000","last_became_peered":"2024-09-19T04:04:46.375386+0000","last_unstale":"2024-09-19T04:04:49.905589+0000","last_undegraded":"2024-09-19T04:04:49.905589+0000","last_fullsized":"2024-09-19T04:04:49.905589+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:39.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884994+0000","last_change":"2024-09-19T04:04:39.329480+0000","last_active":"2024-09-19T04:04:49.884994+0000","last_peered":"2024-09-19T04:04:49.884994+0000","last_clean":"2024-09-19T04:04:49.884994+0000","last_became_active":"2024-09-19T04:04:39.329008+0000","last_became_peered":"2024-09-19T04:04:39.329008+0000","last_unstale":"2024-09-19T04:04:49.884994+0000","last_undegraded":"2024-09-19T04:04:49.884994+0000","last_fullsized":"2024-09-19T04:04:49.884994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:06.782579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.608678+0000","last_change":"2024-09-19T04:04:37.216224+0000","last_active":"2024-09-19T04:05:22.608678+0000","last_peered":"2024-09-19T04:05:22.608678+0000","last_clean":"2024-09-19T04:05:22.608678+0000","last_became_active":"2024-09-19T04:04:37.215846+0000","last_became_peered":"2024-09-19T04:04:37.215846+0000","last_unstale":"2024-09-19T04:05:22.608678+0000","last_undegraded":"2024-09-19T04:05:22.608678+0000","last_fullsized":"2024-09-19T04:05:22.608678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:57.829694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905099+0000","last_change":"2024-09-19T04:03:25.869634+0000","last_active":"2024-09-19T04:04:49.905099+0000","last_peered":"2024-09-19T04:04:49.905099+0000","last_clean":"2024-09-19T04:04:49.905099+0000","last_became_active":"2024-09-19T04:03:25.869250+0000","last_became_peered":"2024-09-19T04:03:25.869250+0000","last_unstale":"2024-09-19T04:04:49.905099+0000","last_undegraded":"2024-09-19T04:04:49.905099+0000","last_fullsized":"2024-09-19T04:04:49.905099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_clean_scrub_stamp":"2024-09-19T03:59:59.253417+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:51.956932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905834+0000","last_change":"2024-09-19T04:04:46.357123+0000","last_active":"2024-09-19T04:04:49.905834+0000","last_peered":"2024-09-19T04:04:49.905834+0000","last_clean":"2024-09-19T04:04:49.905834+0000","last_became_active":"2024-09-19T04:04:46.356671+0000","last_became_peered":"2024-09-19T04:04:46.356671+0000","last_unstale":"2024-09-19T04:04:49.905834+0000","last_undegraded":"2024-09-19T04:04:49.905834+0000","last_fullsized":"2024-09-19T04:04:49.905834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:09.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905969+0000","last_change":"2024-09-19T04:04:39.322487+0000","last_active":"2024-09-19T04:04:49.905969+0000","last_peered":"2024-09-19T04:04:49.905969+0000","last_clean":"2024-09-19T04:04:49.905969+0000","last_became_active":"2024-09-19T04:04:39.321994+0000","last_became_peered":"2024-09-19T04:04:39.321994+0000","last_unstale":"2024-09-19T04:04:49.905969+0000","last_undegraded":"2024-09-19T04:04:49.905969+0000","last_fullsized":"2024-09-19T04:04:49.905969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:56.977405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:31.622545+0000","last_change":"2024-09-19T04:04:37.703452+0000","last_active":"2024-09-19T04:05:31.622545+0000","last_peered":"2024-09-19T04:05:31.622545+0000","last_clean":"2024-09-19T04:05:31.622545+0000","last_became_active":"2024-09-19T04:04:37.703048+0000","last_became_peered":"2024-09-19T04:04:37.703048+0000","last_unstale":"2024-09-19T04:05:31.622545+0000","last_undegraded":"2024-09-19T04:05:31.622545+0000","last_fullsized":"2024-09-19T04:05:31.622545+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:16:29.051617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906348+0000","last_change":"2024-09-19T04:04:46.361229+0000","last_active":"2024-09-19T04:04:49.906348+0000","last_peered":"2024-09-19T04:04:49.906348+0000","last_clean":"2024-09-19T04:04:49.906348+0000","last_became_active":"2024-09-19T04:04:46.360729+0000","last_became_peered":"2024-09-19T04:04:46.360729+0000","last_unstale":"2024-09-19T04:04:49.906348+0000","last_undegraded":"2024-09-19T04:04:49.906348+0000","last_fullsized":"2024-09-19T04:04:49.906348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:48.022778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905796+0000","last_change":"2024-09-19T04:04:39.328598+0000","last_active":"2024-09-19T04:04:49.905796+0000","last_peered":"2024-09-19T04:04:49.905796+0000","last_clean":"2024-09-19T04:04:49.905796+0000","last_became_active":"2024-09-19T04:04:39.328247+0000","last_became_peered":"2024-09-19T04:04:39.328247+0000","last_unstale":"2024-09-19T04:04:49.905796+0000","last_undegraded":"2024-09-19T04:04:49.905796+0000","last_fullsized":"2024-09-19T04:04:49.905796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:34.916487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529955+0000","last_change":"2024-09-19T04:04:37.212065+0000","last_active":"2024-09-19T04:05:22.529955+0000","last_peered":"2024-09-19T04:05:22.529955+0000","last_clean":"2024-09-19T04:05:22.529955+0000","last_became_active":"2024-09-19T04:04:37.211520+0000","last_became_peered":"2024-09-19T04:04:37.211520+0000","last_unstale":"2024-09-19T04:05:22.529955+0000","last_undegraded":"2024-09-19T04:05:22.529955+0000","last_fullsized":"2024-09-19T04:05:22.529955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:53.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905684+0000","last_change":"2024-09-19T04:04:46.358558+0000","last_active":"2024-09-19T04:04:49.905684+0000","last_peered":"2024-09-19T04:04:49.905684+0000","last_clean":"2024-09-19T04:04:49.905684+0000","last_became_active":"2024-09-19T04:04:46.358158+0000","last_became_peered":"2024-09-19T04:04:46.358158+0000","last_unstale":"2024-09-19T04:04:49.905684+0000","last_undegraded":"2024-09-19T04:04:49.905684+0000","last_fullsized":"2024-09-19T04:04:49.905684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:16.627051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906045+0000","last_change":"2024-09-19T04:04:39.325187+0000","last_active":"2024-09-19T04:04:49.906045+0000","last_peered":"2024-09-19T04:04:49.906045+0000","last_clean":"2024-09-19T04:04:49.906045+0000","last_became_active":"2024-09-19T04:04:39.324644+0000","last_became_peered":"2024-09-19T04:04:39.324644+0000","last_unstale":"2024-09-19T04:04:49.906045+0000","last_undegraded":"2024-09-19T04:04:49.906045+0000","last_fullsized":"2024-09-19T04:04:49.906045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:11.879273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.528447+0000","last_change":"2024-09-19T04:04:37.205389+0000","last_active":"2024-09-19T04:05:22.528447+0000","last_peered":"2024-09-19T04:05:22.528447+0000","last_clean":"2024-09-19T04:05:22.528447+0000","last_became_active":"2024-09-19T04:04:37.205117+0000","last_became_peered":"2024-09-19T04:04:37.205117+0000","last_unstale":"2024-09-19T04:05:22.528447+0000","last_undegraded":"2024-09-19T04:05:22.528447+0000","last_fullsized":"2024-09-19T04:05:22.528447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:49.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904546+0000","last_change":"2024-09-19T04:04:46.359815+0000","last_active":"2024-09-19T04:04:49.904546+0000","last_peered":"2024-09-19T04:04:49.904546+0000","last_clean":"2024-09-19T04:04:49.904546+0000","last_became_active":"2024-09-19T04:04:46.359185+0000","last_became_peered":"2024-09-19T04:04:46.359185+0000","last_unstale":"2024-09-19T04:04:49.904546+0000","last_undegraded":"2024-09-19T04:04:49.904546+0000","last_fullsized":"2024-09-19T04:04:49.904546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:54.872722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905210+0000","last_change":"2024-09-19T04:04:39.328841+0000","last_active":"2024-09-19T04:04:49.905210+0000","last_peered":"2024-09-19T04:04:49.905210+0000","last_clean":"2024-09-19T04:04:49.905210+0000","last_became_active":"2024-09-19T04:04:39.328470+0000","last_became_peered":"2024-09-19T04:04:39.328470+0000","last_unstale":"2024-09-19T04:04:49.905210+0000","last_undegraded":"2024-09-19T04:04:49.905210+0000","last_fullsized":"2024-09-19T04:04:49.905210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:17.701469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:21.504227+0000","last_change":"2024-09-19T04:04:37.208116+0000","last_active":"2024-09-19T04:05:21.504227+0000","last_peered":"2024-09-19T04:05:21.504227+0000","last_clean":"2024-09-19T04:05:21.504227+0000","last_became_active":"2024-09-19T04:04:37.207720+0000","last_became_peered":"2024-09-19T04:04:37.207720+0000","last_unstale":"2024-09-19T04:05:21.504227+0000","last_undegraded":"2024-09-19T04:05:21.504227+0000","last_fullsized":"2024-09-19T04:05:21.504227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:01.191545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905195+0000","last_change":"2024-09-19T04:04:46.361199+0000","last_active":"2024-09-19T04:04:49.905195+0000","last_peered":"2024-09-19T04:04:49.905195+0000","last_clean":"2024-09-19T04:04:49.905195+0000","last_became_active":"2024-09-19T04:04:46.360639+0000","last_became_peered":"2024-09-19T04:04:46.360639+0000","last_unstale":"2024-09-19T04:04:49.905195+0000","last_undegraded":"2024-09-19T04:04:49.905195+0000","last_fullsized":"2024-09-19T04:04:49.905195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:50.185760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905226+0000","last_change":"2024-09-19T04:04:39.328529+0000","last_active":"2024-09-19T04:04:49.905226+0000","last_peered":"2024-09-19T04:04:49.905226+0000","last_clean":"2024-09-19T04:04:49.905226+0000","last_became_active":"2024-09-19T04:04:39.328174+0000","last_became_peered":"2024-09-19T04:04:39.328174+0000","last_unstale":"2024-09-19T04:04:49.905226+0000","last_undegraded":"2024-09-19T04:04:49.905226+0000","last_fullsized":"2024-09-19T04:04:49.905226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:02.503672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.530341+0000","last_change":"2024-09-19T04:04:37.647919+0000","last_active":"2024-09-19T04:05:22.530341+0000","last_peered":"2024-09-19T04:05:22.530341+0000","last_clean":"2024-09-19T04:05:22.530341+0000","last_became_active":"2024-09-19T04:04:37.647305+0000","last_became_peered":"2024-09-19T04:04:37.647305+0000","last_unstale":"2024-09-19T04:05:22.530341+0000","last_undegraded":"2024-09-19T04:05:22.530341+0000","last_fullsized":"2024-09-19T04:05:22.530341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:07.035465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905134+0000","last_change":"2024-09-19T04:04:46.360362+0000","last_active":"2024-09-19T04:04:49.905134+0000","last_peered":"2024-09-19T04:04:49.905134+0000","last_clean":"2024-09-19T04:04:49.905134+0000","last_became_active":"2024-09-19T04:04:46.359470+0000","last_became_peered":"2024-09-19T04:04:46.359470+0000","last_unstale":"2024-09-19T04:04:49.905134+0000","last_undegraded":"2024-09-19T04:04:49.905134+0000","last_fullsized":"2024-09-19T04:04:49.905134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:43.804073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904892+0000","last_change":"2024-09-19T04:04:39.326090+0000","last_active":"2024-09-19T04:04:49.904892+0000","last_peered":"2024-09-19T04:04:49.904892+0000","last_clean":"2024-09-19T04:04:49.904892+0000","last_became_active":"2024-09-19T04:04:39.325751+0000","last_became_peered":"2024-09-19T04:04:39.325751+0000","last_unstale":"2024-09-19T04:04:49.904892+0000","last_undegraded":"2024-09-19T04:04:49.904892+0000","last_fullsized":"2024-09-19T04:04:49.904892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:25.357467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:33.625266+0000","last_change":"2024-09-19T04:04:37.215107+0000","last_active":"2024-09-19T04:05:33.625266+0000","last_peered":"2024-09-19T04:05:33.625266+0000","last_clean":"2024-09-19T04:05:33.625266+0000","last_became_active":"2024-09-19T04:04:37.214693+0000","last_became_peered":"2024-09-19T04:04:37.214693+0000","last_unstale":"2024-09-19T04:05:33.625266+0000","last_undegraded":"2024-09-19T04:05:33.625266+0000","last_fullsized":"2024-09-19T04:05:33.625266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:56.535316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350946+0000","last_change":"2024-09-19T04:04:46.363625+0000","last_active":"2024-09-19T04:04:49.350946+0000","last_peered":"2024-09-19T04:04:49.350946+0000","last_clean":"2024-09-19T04:04:49.350946+0000","last_became_active":"2024-09-19T04:04:46.363207+0000","last_became_peered":"2024-09-19T04:04:46.363207+0000","last_unstale":"2024-09-19T04:04:49.350946+0000","last_undegraded":"2024-09-19T04:04:49.350946+0000","last_fullsized":"2024-09-19T04:04:49.350946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:30.251168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905446+0000","last_change":"2024-09-19T04:04:39.317977+0000","last_active":"2024-09-19T04:04:49.905446+0000","last_peered":"2024-09-19T04:04:49.905446+0000","last_clean":"2024-09-19T04:04:49.905446+0000","last_became_active":"2024-09-19T04:04:39.317658+0000","last_became_peered":"2024-09-19T04:04:39.317658+0000","last_unstale":"2024-09-19T04:04:49.905446+0000","last_undegraded":"2024-09-19T04:04:49.905446+0000","last_fullsized":"2024-09-19T04:04:49.905446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:34.370854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529778+0000","last_change":"2024-09-19T04:04:37.212103+0000","last_active":"2024-09-19T04:05:22.529778+0000","last_peered":"2024-09-19T04:05:22.529778+0000","last_clean":"2024-09-19T04:05:22.529778+0000","last_became_active":"2024-09-19T04:04:37.211739+0000","last_became_peered":"2024-09-19T04:04:37.211739+0000","last_unstale":"2024-09-19T04:05:22.529778+0000","last_undegraded":"2024-09-19T04:05:22.529778+0000","last_fullsized":"2024-09-19T04:05:22.529778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:06.588194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904937+0000","last_change":"2024-09-19T04:04:46.361975+0000","last_active":"2024-09-19T04:04:49.904937+0000","last_peered":"2024-09-19T04:04:49.904937+0000","last_clean":"2024-09-19T04:04:49.904937+0000","last_became_active":"2024-09-19T04:04:46.361492+0000","last_became_peered":"2024-09-19T04:04:46.361492+0000","last_unstale":"2024-09-19T04:04:49.904937+0000","last_undegraded":"2024-09-19T04:04:49.904937+0000","last_fullsized":"2024-09-19T04:04:49.904937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:52.723836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904429+0000","last_change":"2024-09-19T04:04:39.317756+0000","last_active":"2024-09-19T04:04:49.904429+0000","last_peered":"2024-09-19T04:04:49.904429+0000","last_clean":"2024-09-19T04:04:49.904429+0000","last_became_active":"2024-09-19T04:04:39.317322+0000","last_became_peered":"2024-09-19T04:04:39.317322+0000","last_unstale":"2024-09-19T04:04:49.904429+0000","last_undegraded":"2024-09-19T04:04:49.904429+0000","last_fullsized":"2024-09-19T04:04:49.904429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:26.291574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.606813+0000","last_change":"2024-09-19T04:04:37.213892+0000","last_active":"2024-09-19T04:05:22.606813+0000","last_peered":"2024-09-19T04:05:22.606813+0000","last_clean":"2024-09-19T04:05:22.606813+0000","last_became_active":"2024-09-19T04:04:37.213350+0000","last_became_peered":"2024-09-19T04:04:37.213350+0000","last_unstale":"2024-09-19T04:05:22.606813+0000","last_undegraded":"2024-09-19T04:05:22.606813+0000","last_fullsized":"2024-09-19T04:05:22.606813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:40.065717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905896+0000","last_change":"2024-09-19T04:04:46.360566+0000","last_active":"2024-09-19T04:04:49.905896+0000","last_peered":"2024-09-19T04:04:49.905896+0000","last_clean":"2024-09-19T04:04:49.905896+0000","last_became_active":"2024-09-19T04:04:46.360143+0000","last_became_peered":"2024-09-19T04:04:46.360143+0000","last_unstale":"2024-09-19T04:04:49.905896+0000","last_undegraded":"2024-09-19T04:04:49.905896+0000","last_fullsized":"2024-09-19T04:04:49.905896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:59.997088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905190+0000","last_change":"2024-09-19T04:04:39.318363+0000","last_active":"2024-09-19T04:04:49.905190+0000","last_peered":"2024-09-19T04:04:49.905190+0000","last_clean":"2024-09-19T04:04:49.905190+0000","last_became_active":"2024-09-19T04:04:39.318003+0000","last_became_peered":"2024-09-19T04:04:39.318003+0000","last_unstale":"2024-09-19T04:04:49.905190+0000","last_undegraded":"2024-09-19T04:04:49.905190+0000","last_fullsized":"2024-09-19T04:04:49.905190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:54.128795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'27","reported_seq":84,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:32.626279+0000","last_change":"2024-09-19T04:04:37.212614+0000","last_active":"2024-09-19T04:05:32.626279+0000","last_peered":"2024-09-19T04:05:32.626279+0000","last_clean":"2024-09-19T04:05:32.626279+0000","last_became_active":"2024-09-19T04:04:37.212153+0000","last_became_peered":"2024-09-19T04:04:37.212153+0000","last_unstale":"2024-09-19T04:05:32.626279+0000","last_undegraded":"2024-09-19T04:05:32.626279+0000","last_fullsized":"2024-09-19T04:05:32.626279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:11.930287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":56,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350608+0000","last_change":"2024-09-19T04:04:46.362967+0000","last_active":"2024-09-19T04:04:49.350608+0000","last_peered":"2024-09-19T04:04:49.350608+0000","last_clean":"2024-09-19T04:04:49.350608+0000","last_became_active":"2024-09-19T04:04:46.362529+0000","last_became_peered":"2024-09-19T04:04:46.362529+0000","last_unstale":"2024-09-19T04:04:49.350608+0000","last_undegraded":"2024-09-19T04:04:49.350608+0000","last_fullsized":"2024-09-19T04:04:49.350608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:01.821697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905771+0000","last_change":"2024-09-19T04:04:39.323499+0000","last_active":"2024-09-19T04:04:49.905771+0000","last_peered":"2024-09-19T04:04:49.905771+0000","last_clean":"2024-09-19T04:04:49.905771+0000","last_became_active":"2024-09-19T04:04:39.323076+0000","last_became_peered":"2024-09-19T04:04:39.323076+0000","last_unstale":"2024-09-19T04:04:49.905771+0000","last_undegraded":"2024-09-19T04:04:49.905771+0000","last_fullsized":"2024-09-19T04:04:49.905771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:10.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:21.513796+0000","last_change":"2024-09-19T04:04:37.647927+0000","last_active":"2024-09-19T04:05:21.513796+0000","last_peered":"2024-09-19T04:05:21.513796+0000","last_clean":"2024-09-19T04:05:21.513796+0000","last_became_active":"2024-09-19T04:04:37.647279+0000","last_became_peered":"2024-09-19T04:04:37.647279+0000","last_unstale":"2024-09-19T04:05:21.513796+0000","last_undegraded":"2024-09-19T04:05:21.513796+0000","last_fullsized":"2024-09-19T04:05:21.513796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:05:50.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904874+0000","last_change":"2024-09-19T04:04:46.363562+0000","last_active":"2024-09-19T04:04:49.904874+0000","last_peered":"2024-09-19T04:04:49.904874+0000","last_clean":"2024-09-19T04:04:49.904874+0000","last_became_active":"2024-09-19T04:04:46.362915+0000","last_became_peered":"2024-09-19T04:04:46.362915+0000","last_unstale":"2024-09-19T04:04:49.904874+0000","last_undegraded":"2024-09-19T04:04:49.904874+0000","last_fullsized":"2024-09-19T04:04:49.904874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:08.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905947+0000","last_change":"2024-09-19T04:04:39.329034+0000","last_active":"2024-09-19T04:04:49.905947+0000","last_peered":"2024-09-19T04:04:49.905947+0000","last_clean":"2024-09-19T04:04:49.905947+0000","last_became_active":"2024-09-19T04:04:39.328692+0000","last_became_peered":"2024-09-19T04:04:39.328692+0000","last_unstale":"2024-09-19T04:04:49.905947+0000","last_undegraded":"2024-09-19T04:04:49.905947+0000","last_fullsized":"2024-09-19T04:04:49.905947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:42.903089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529488+0000","last_change":"2024-09-19T04:04:37.212393+0000","last_active":"2024-09-19T04:05:22.529488+0000","last_peered":"2024-09-19T04:05:22.529488+0000","last_clean":"2024-09-19T04:05:22.529488+0000","last_became_active":"2024-09-19T04:04:37.212013+0000","last_became_peered":"2024-09-19T04:04:37.212013+0000","last_unstale":"2024-09-19T04:05:22.529488+0000","last_undegraded":"2024-09-19T04:05:22.529488+0000","last_fullsized":"2024-09-19T04:05:22.529488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:37.981075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350389+0000","last_change":"2024-09-19T04:04:46.359239+0000","last_active":"2024-09-19T04:04:49.350389+0000","last_peered":"2024-09-19T04:04:49.350389+0000","last_clean":"2024-09-19T04:04:49.350389+0000","last_became_active":"2024-09-19T04:04:46.358881+0000","last_became_peered":"2024-09-19T04:04:46.358881+0000","last_unstale":"2024-09-19T04:04:49.350389+0000","last_undegraded":"2024-09-19T04:04:49.350389+0000","last_fullsized":"2024-09-19T04:04:49.350389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:58.379187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906237+0000","last_change":"2024-09-19T04:04:39.319705+0000","last_active":"2024-09-19T04:04:49.906237+0000","last_peered":"2024-09-19T04:04:49.906237+0000","last_clean":"2024-09-19T04:04:49.906237+0000","last_became_active":"2024-09-19T04:04:39.318065+0000","last_became_peered":"2024-09-19T04:04:39.318065+0000","last_unstale":"2024-09-19T04:04:49.906237+0000","last_undegraded":"2024-09-19T04:04:49.906237+0000","last_fullsized":"2024-09-19T04:04:49.906237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:15.630927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:32.623156+0000","last_change":"2024-09-19T04:04:37.215192+0000","last_active":"2024-09-19T04:05:32.623156+0000","last_peered":"2024-09-19T04:05:32.623156+0000","last_clean":"2024-09-19T04:05:32.623156+0000","last_became_active":"2024-09-19T04:04:37.214872+0000","last_became_peered":"2024-09-19T04:04:37.214872+0000","last_unstale":"2024-09-19T04:05:32.623156+0000","last_undegraded":"2024-09-19T04:05:32.623156+0000","last_fullsized":"2024-09-19T04:05:32.623156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:18.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884852+0000","last_change":"2024-09-19T04:04:46.358113+0000","last_active":"2024-09-19T04:04:49.884852+0000","last_peered":"2024-09-19T04:04:49.884852+0000","last_clean":"2024-09-19T04:04:49.884852+0000","last_became_active":"2024-09-19T04:04:46.356604+0000","last_became_peered":"2024-09-19T04:04:46.356604+0000","last_unstale":"2024-09-19T04:04:49.884852+0000","last_undegraded":"2024-09-19T04:04:49.884852+0000","last_fullsized":"2024-09-19T04:04:49.884852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:38.147738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905726+0000","last_change":"2024-09-19T04:04:46.354516+0000","last_active":"2024-09-19T04:04:49.905726+0000","last_peered":"2024-09-19T04:04:49.905726+0000","last_clean":"2024-09-19T04:04:49.905726+0000","last_became_active":"2024-09-19T04:04:46.354034+0000","last_became_peered":"2024-09-19T04:04:46.354034+0000","last_unstale":"2024-09-19T04:04:49.905726+0000","last_undegraded":"2024-09-19T04:04:49.905726+0000","last_fullsized":"2024-09-19T04:04:49.905726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:21.946746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350792+0000","last_change":"2024-09-19T04:04:39.311324+0000","last_active":"2024-09-19T04:04:49.350792+0000","last_peered":"2024-09-19T04:04:49.350792+0000","last_clean":"2024-09-19T04:04:49.350792+0000","last_became_active":"2024-09-19T04:04:39.310977+0000","last_became_peered":"2024-09-19T04:04:39.310977+0000","last_unstale":"2024-09-19T04:04:49.350792+0000","last_undegraded":"2024-09-19T04:04:49.350792+0000","last_fullsized":"2024-09-19T04:04:49.350792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:43.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.530118+0000","last_change":"2024-09-19T04:04:37.212773+0000","last_active":"2024-09-19T04:05:22.530118+0000","last_peered":"2024-09-19T04:05:22.530118+0000","last_clean":"2024-09-19T04:05:22.530118+0000","last_became_active":"2024-09-19T04:04:37.212405+0000","last_became_peered":"2024-09-19T04:04:37.212405+0000","last_unstale":"2024-09-19T04:05:22.530118+0000","last_undegraded":"2024-09-19T04:05:22.530118+0000","last_fullsized":"2024-09-19T04:05:22.530118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:29.020122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905300+0000","last_change":"2024-09-19T04:04:46.363052+0000","last_active":"2024-09-19T04:04:49.905300+0000","last_peered":"2024-09-19T04:04:49.905300+0000","last_clean":"2024-09-19T04:04:49.905300+0000","last_became_active":"2024-09-19T04:04:46.362588+0000","last_became_peered":"2024-09-19T04:04:46.362588+0000","last_unstale":"2024-09-19T04:04:49.905300+0000","last_undegraded":"2024-09-19T04:04:49.905300+0000","last_fullsized":"2024-09-19T04:04:49.905300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:52.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885530+0000","last_change":"2024-09-19T04:04:39.313686+0000","last_active":"2024-09-19T04:04:49.885530+0000","last_peered":"2024-09-19T04:04:49.885530+0000","last_clean":"2024-09-19T04:04:49.885530+0000","last_became_active":"2024-09-19T04:04:39.313386+0000","last_became_peered":"2024-09-19T04:04:39.313386+0000","last_unstale":"2024-09-19T04:04:49.885530+0000","last_undegraded":"2024-09-19T04:04:49.885530+0000","last_fullsized":"2024-09-19T04:04:49.885530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:08.274930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:20.508029+0000","last_change":"2024-09-19T04:04:37.207926+0000","last_active":"2024-09-19T04:05:20.508029+0000","last_peered":"2024-09-19T04:05:20.508029+0000","last_clean":"2024-09-19T04:05:20.508029+0000","last_became_active":"2024-09-19T04:04:37.207538+0000","last_became_peered":"2024-09-19T04:04:37.207538+0000","last_unstale":"2024-09-19T04:05:20.508029+0000","last_undegraded":"2024-09-19T04:05:20.508029+0000","last_fullsized":"2024-09-19T04:05:20.508029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:29.621638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:21.514391+0000","last_change":"2024-09-19T04:04:37.212829+0000","last_active":"2024-09-19T04:05:21.514391+0000","last_peered":"2024-09-19T04:05:21.514391+0000","last_clean":"2024-09-19T04:05:21.514391+0000","last_became_active":"2024-09-19T04:04:37.212294+0000","last_became_peered":"2024-09-19T04:04:37.212294+0000","last_unstale":"2024-09-19T04:05:21.514391+0000","last_undegraded":"2024-09-19T04:05:21.514391+0000","last_fullsized":"2024-09-19T04:05:21.514391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:34.479708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905815+0000","last_change":"2024-09-19T04:04:39.328622+0000","last_active":"2024-09-19T04:04:49.905815+0000","last_peered":"2024-09-19T04:04:49.905815+0000","last_clean":"2024-09-19T04:04:49.905815+0000","last_became_active":"2024-09-19T04:04:39.328290+0000","last_became_peered":"2024-09-19T04:04:39.328290+0000","last_unstale":"2024-09-19T04:04:49.905815+0000","last_undegraded":"2024-09-19T04:04:49.905815+0000","last_fullsized":"2024-09-19T04:04:49.905815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:04.290767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351654+0000","last_change":"2024-09-19T04:04:46.362853+0000","last_active":"2024-09-19T04:04:49.351654+0000","last_peered":"2024-09-19T04:04:49.351654+0000","last_clean":"2024-09-19T04:04:49.351654+0000","last_became_active":"2024-09-19T04:04:46.362392+0000","last_became_peered":"2024-09-19T04:04:46.362392+0000","last_unstale":"2024-09-19T04:04:49.351654+0000","last_undegraded":"2024-09-19T04:04:49.351654+0000","last_fullsized":"2024-09-19T04:04:49.351654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:40.016502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:33.625173+0000","last_change":"2024-09-19T04:04:37.213248+0000","last_active":"2024-09-19T04:05:33.625173+0000","last_peered":"2024-09-19T04:05:33.625173+0000","last_clean":"2024-09-19T04:05:33.625173+0000","last_became_active":"2024-09-19T04:04:37.212639+0000","last_became_peered":"2024-09-19T04:04:37.212639+0000","last_unstale":"2024-09-19T04:05:33.625173+0000","last_undegraded":"2024-09-19T04:05:33.625173+0000","last_fullsized":"2024-09-19T04:05:33.625173+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:45:05.917792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904639+0000","last_change":"2024-09-19T04:04:39.330160+0000","last_active":"2024-09-19T04:04:49.904639+0000","last_peered":"2024-09-19T04:04:49.904639+0000","last_clean":"2024-09-19T04:04:49.904639+0000","last_became_active":"2024-09-19T04:04:39.329828+0000","last_became_peered":"2024-09-19T04:04:39.329828+0000","last_unstale":"2024-09-19T04:04:49.904639+0000","last_undegraded":"2024-09-19T04:04:49.904639+0000","last_fullsized":"2024-09-19T04:04:49.904639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:57.712550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.352046+0000","last_change":"2024-09-19T04:04:46.360459+0000","last_active":"2024-09-19T04:04:49.352046+0000","last_peered":"2024-09-19T04:04:49.352046+0000","last_clean":"2024-09-19T04:04:49.352046+0000","last_became_active":"2024-09-19T04:04:46.359914+0000","last_became_peered":"2024-09-19T04:04:46.359914+0000","last_unstale":"2024-09-19T04:04:49.352046+0000","last_undegraded":"2024-09-19T04:04:49.352046+0000","last_fullsized":"2024-09-19T04:04:49.352046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:30.380358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904393+0000","last_change":"2024-09-19T04:04:39.324847+0000","last_active":"2024-09-19T04:04:49.904393+0000","last_peered":"2024-09-19T04:04:49.904393+0000","last_clean":"2024-09-19T04:04:49.904393+0000","last_became_active":"2024-09-19T04:04:39.324402+0000","last_became_peered":"2024-09-19T04:04:39.324402+0000","last_unstale":"2024-09-19T04:04:49.904393+0000","last_undegraded":"2024-09-19T04:04:49.904393+0000","last_fullsized":"2024-09-19T04:04:49.904393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:41.996135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351893+0000","last_change":"2024-09-19T04:04:39.324906+0000","last_active":"2024-09-19T04:04:49.351893+0000","last_peered":"2024-09-19T04:04:49.351893+0000","last_clean":"2024-09-19T04:04:49.351893+0000","last_became_active":"2024-09-19T04:04:39.324552+0000","last_became_peered":"2024-09-19T04:04:39.324552+0000","last_unstale":"2024-09-19T04:04:49.351893+0000","last_undegraded":"2024-09-19T04:04:49.351893+0000","last_fullsized":"2024-09-19T04:04:49.351893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:32:23.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351472+0000","last_change":"2024-09-19T04:04:39.318556+0000","last_active":"2024-09-19T04:04:49.351472+0000","last_peered":"2024-09-19T04:04:49.351472+0000","last_clean":"2024-09-19T04:04:49.351472+0000","last_became_active":"2024-09-19T04:04:39.318200+0000","last_became_peered":"2024-09-19T04:04:39.318200+0000","last_unstale":"2024-09-19T04:04:49.351472+0000","last_undegraded":"2024-09-19T04:04:49.351472+0000","last_fullsized":"2024-09-19T04:04:49.351472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:40.847440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905722+0000","last_change":"2024-09-19T04:04:39.320698+0000","last_active":"2024-09-19T04:04:49.905722+0000","last_peered":"2024-09-19T04:04:49.905722+0000","last_clean":"2024-09-19T04:04:49.905722+0000","last_became_active":"2024-09-19T04:04:39.320419+0000","last_became_peered":"2024-09-19T04:04:39.320419+0000","last_unstale":"2024-09-19T04:04:49.905722+0000","last_undegraded":"2024-09-19T04:04:49.905722+0000","last_fullsized":"2024-09-19T04:04:49.905722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:32.708394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904906+0000","last_change":"2024-09-19T04:04:39.319886+0000","last_active":"2024-09-19T04:04:49.904906+0000","last_peered":"2024-09-19T04:04:49.904906+0000","last_clean":"2024-09-19T04:04:49.904906+0000","last_became_active":"2024-09-19T04:04:39.319217+0000","last_became_peered":"2024-09-19T04:04:39.319217+0000","last_unstale":"2024-09-19T04:04:49.904906+0000","last_undegraded":"2024-09-19T04:04:49.904906+0000","last_fullsized":"2024-09-19T04:04:49.904906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:38.829608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905013+0000","last_change":"2024-09-19T04:04:39.327639+0000","last_active":"2024-09-19T04:04:49.905013+0000","last_peered":"2024-09-19T04:04:49.905013+0000","last_clean":"2024-09-19T04:04:49.905013+0000","last_became_active":"2024-09-19T04:04:39.327349+0000","last_became_peered":"2024-09-19T04:04:39.327349+0000","last_unstale":"2024-09-19T04:04:49.905013+0000","last_undegraded":"2024-09-19T04:04:49.905013+0000","last_fullsized":"2024-09-19T04:04:49.905013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:31.056609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.352008+0000","last_change":"2024-09-19T04:04:39.320253+0000","last_active":"2024-09-19T04:04:49.352008+0000","last_peered":"2024-09-19T04:04:49.352008+0000","last_clean":"2024-09-19T04:04:49.352008+0000","last_became_active":"2024-09-19T04:04:39.319851+0000","last_became_peered":"2024-09-19T04:04:39.319851+0000","last_unstale":"2024-09-19T04:04:49.352008+0000","last_undegraded":"2024-09-19T04:04:49.352008+0000","last_fullsized":"2024-09-19T04:04:49.352008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:53.631216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885400+0000","last_change":"2024-09-19T04:04:39.322779+0000","last_active":"2024-09-19T04:04:49.885400+0000","last_peered":"2024-09-19T04:04:49.885400+0000","last_clean":"2024-09-19T04:04:49.885400+0000","last_became_active":"2024-09-19T04:04:39.322498+0000","last_became_peered":"2024-09-19T04:04:39.322498+0000","last_unstale":"2024-09-19T04:04:49.885400+0000","last_undegraded":"2024-09-19T04:04:49.885400+0000","last_fullsized":"2024-09-19T04:04:49.885400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:24:11.956002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904389+0000","last_change":"2024-09-19T04:04:39.322759+0000","last_active":"2024-09-19T04:04:49.904389+0000","last_peered":"2024-09-19T04:04:49.904389+0000","last_clean":"2024-09-19T04:04:49.904389+0000","last_became_active":"2024-09-19T04:04:39.322429+0000","last_became_peered":"2024-09-19T04:04:39.322429+0000","last_unstale":"2024-09-19T04:04:49.904389+0000","last_undegraded":"2024-09-19T04:04:49.904389+0000","last_fullsized":"2024-09-19T04:04:49.904389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:36.069931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884890+0000","last_change":"2024-09-19T04:04:39.320078+0000","last_active":"2024-09-19T04:04:49.884890+0000","last_peered":"2024-09-19T04:04:49.884890+0000","last_clean":"2024-09-19T04:04:49.884890+0000","last_became_active":"2024-09-19T04:04:39.319755+0000","last_became_peered":"2024-09-19T04:04:39.319755+0000","last_unstale":"2024-09-19T04:04:49.884890+0000","last_undegraded":"2024-09-19T04:04:49.884890+0000","last_fullsized":"2024-09-19T04:04:49.884890+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:07.651712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906472+0000","last_change":"2024-09-19T04:04:39.319839+0000","last_active":"2024-09-19T04:04:49.906472+0000","last_peered":"2024-09-19T04:04:49.906472+0000","last_clean":"2024-09-19T04:04:49.906472+0000","last_became_active":"2024-09-19T04:04:39.319527+0000","last_became_peered":"2024-09-19T04:04:39.319527+0000","last_unstale":"2024-09-19T04:04:49.906472+0000","last_undegraded":"2024-09-19T04:04:49.906472+0000","last_fullsized":"2024-09-19T04:04:49.906472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:31.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904823+0000","last_change":"2024-09-19T04:04:39.315975+0000","last_active":"2024-09-19T04:04:49.904823+0000","last_peered":"2024-09-19T04:04:49.904823+0000","last_clean":"2024-09-19T04:04:49.904823+0000","last_became_active":"2024-09-19T04:04:39.315671+0000","last_became_peered":"2024-09-19T04:04:39.315671+0000","last_unstale":"2024-09-19T04:04:49.904823+0000","last_undegraded":"2024-09-19T04:04:49.904823+0000","last_fullsized":"2024-09-19T04:04:49.904823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:16.349853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905047+0000","last_change":"2024-09-19T04:04:39.316278+0000","last_active":"2024-09-19T04:04:49.905047+0000","last_peered":"2024-09-19T04:04:49.905047+0000","last_clean":"2024-09-19T04:04:49.905047+0000","last_became_active":"2024-09-19T04:04:39.315876+0000","last_became_peered":"2024-09-19T04:04:39.315876+0000","last_unstale":"2024-09-19T04:04:49.905047+0000","last_undegraded":"2024-09-19T04:04:49.905047+0000","last_fullsized":"2024-09-19T04:04:49.905047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:23.544405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.352016+0000","last_change":"2024-09-19T04:04:39.318952+0000","last_active":"2024-09-19T04:04:49.352016+0000","last_peered":"2024-09-19T04:04:49.352016+0000","last_clean":"2024-09-19T04:04:49.352016+0000","last_became_active":"2024-09-19T04:04:39.318598+0000","last_became_peered":"2024-09-19T04:04:39.318598+0000","last_unstale":"2024-09-19T04:04:49.352016+0000","last_undegraded":"2024-09-19T04:04:49.352016+0000","last_fullsized":"2024-09-19T04:04:49.352016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:12.654611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350931+0000","last_change":"2024-09-19T04:04:39.311334+0000","last_active":"2024-09-19T04:04:49.350931+0000","last_peered":"2024-09-19T04:04:49.350931+0000","last_clean":"2024-09-19T04:04:49.350931+0000","last_became_active":"2024-09-19T04:04:39.311038+0000","last_became_peered":"2024-09-19T04:04:39.311038+0000","last_unstale":"2024-09-19T04:04:49.350931+0000","last_undegraded":"2024-09-19T04:04:49.350931+0000","last_fullsized":"2024-09-19T04:04:49.350931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:41.655363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905689+0000","last_change":"2024-09-19T04:04:39.319529+0000","last_active":"2024-09-19T04:04:49.905689+0000","last_peered":"2024-09-19T04:04:49.905689+0000","last_clean":"2024-09-19T04:04:49.905689+0000","last_became_active":"2024-09-19T04:04:39.318787+0000","last_became_peered":"2024-09-19T04:04:39.318787+0000","last_unstale":"2024-09-19T04:04:49.905689+0000","last_undegraded":"2024-09-19T04:04:49.905689+0000","last_fullsized":"2024-09-19T04:04:49.905689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.209422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904793+0000","last_change":"2024-09-19T04:04:39.320320+0000","last_active":"2024-09-19T04:04:49.904793+0000","last_peered":"2024-09-19T04:04:49.904793+0000","last_clean":"2024-09-19T04:04:49.904793+0000","last_became_active":"2024-09-19T04:04:39.319091+0000","last_became_peered":"2024-09-19T04:04:39.319091+0000","last_unstale":"2024-09-19T04:04:49.904793+0000","last_undegraded":"2024-09-19T04:04:49.904793+0000","last_fullsized":"2024-09-19T04:04:49.904793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:39.495079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905562+0000","last_change":"2024-09-19T04:04:39.323270+0000","last_active":"2024-09-19T04:04:49.905562+0000","last_peered":"2024-09-19T04:04:49.905562+0000","last_clean":"2024-09-19T04:04:49.905562+0000","last_became_active":"2024-09-19T04:04:39.322890+0000","last_became_peered":"2024-09-19T04:04:39.322890+0000","last_unstale":"2024-09-19T04:04:49.905562+0000","last_undegraded":"2024-09-19T04:04:49.905562+0000","last_fullsized":"2024-09-19T04:04:49.905562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:07.311654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905271+0000","last_change":"2024-09-19T04:04:39.317098+0000","last_active":"2024-09-19T04:04:49.905271+0000","last_peered":"2024-09-19T04:04:49.905271+0000","last_clean":"2024-09-19T04:04:49.905271+0000","last_became_active":"2024-09-19T04:04:39.316792+0000","last_became_peered":"2024-09-19T04:04:39.316792+0000","last_unstale":"2024-09-19T04:04:49.905271+0000","last_undegraded":"2024-09-19T04:04:49.905271+0000","last_fullsized":"2024-09-19T04:04:49.905271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.571149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904697+0000","last_change":"2024-09-19T04:04:39.326473+0000","last_active":"2024-09-19T04:04:49.904697+0000","last_peered":"2024-09-19T04:04:49.904697+0000","last_clean":"2024-09-19T04:04:49.904697+0000","last_became_active":"2024-09-19T04:04:39.325506+0000","last_became_peered":"2024-09-19T04:04:39.325506+0000","last_unstale":"2024-09-19T04:04:49.904697+0000","last_undegraded":"2024-09-19T04:04:49.904697+0000","last_fullsized":"2024-09-19T04:04:49.904697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:40.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905674+0000","last_change":"2024-09-19T04:04:39.323536+0000","last_active":"2024-09-19T04:04:49.905674+0000","last_peered":"2024-09-19T04:04:49.905674+0000","last_clean":"2024-09-19T04:04:49.905674+0000","last_became_active":"2024-09-19T04:04:39.323224+0000","last_became_peered":"2024-09-19T04:04:39.323224+0000","last_unstale":"2024-09-19T04:04:49.905674+0000","last_undegraded":"2024-09-19T04:04:49.905674+0000","last_fullsized":"2024-09-19T04:04:49.905674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:54.640599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906115+0000","last_change":"2024-09-19T04:04:39.323802+0000","last_active":"2024-09-19T04:04:49.906115+0000","last_peered":"2024-09-19T04:04:49.906115+0000","last_clean":"2024-09-19T04:04:49.906115+0000","last_became_active":"2024-09-19T04:04:39.323031+0000","last_became_peered":"2024-09-19T04:04:39.323031+0000","last_unstale":"2024-09-19T04:04:49.906115+0000","last_undegraded":"2024-09-19T04:04:49.906115+0000","last_fullsized":"2024-09-19T04:04:49.906115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:12.467182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351988+0000","last_change":"2024-09-19T04:04:39.324870+0000","last_active":"2024-09-19T04:04:49.351988+0000","last_peered":"2024-09-19T04:04:49.351988+0000","last_clean":"2024-09-19T04:04:49.351988+0000","last_became_active":"2024-09-19T04:04:39.324551+0000","last_became_peered":"2024-09-19T04:04:39.324551+0000","last_unstale":"2024-09-19T04:04:49.351988+0000","last_undegraded":"2024-09-19T04:04:49.351988+0000","last_fullsized":"2024-09-19T04:04:49.351988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:34.766437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905559+0000","last_change":"2024-09-19T04:04:39.329086+0000","last_active":"2024-09-19T04:04:49.905559+0000","last_peered":"2024-09-19T04:04:49.905559+0000","last_clean":"2024-09-19T04:04:49.905559+0000","last_became_active":"2024-09-19T04:04:39.328651+0000","last_became_peered":"2024-09-19T04:04:39.328651+0000","last_unstale":"2024-09-19T04:04:49.905559+0000","last_undegraded":"2024-09-19T04:04:49.905559+0000","last_fullsized":"2024-09-19T04:04:49.905559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:12.079162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885021+0000","last_change":"2024-09-19T04:04:39.325427+0000","last_active":"2024-09-19T04:04:49.885021+0000","last_peered":"2024-09-19T04:04:49.885021+0000","last_clean":"2024-09-19T04:04:49.885021+0000","last_became_active":"2024-09-19T04:04:39.325113+0000","last_became_peered":"2024-09-19T04:04:39.325113+0000","last_unstale":"2024-09-19T04:04:49.885021+0000","last_undegraded":"2024-09-19T04:04:49.885021+0000","last_fullsized":"2024-09-19T04:04:49.885021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:15.844503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905172+0000","last_change":"2024-09-19T04:04:39.325032+0000","last_active":"2024-09-19T04:04:49.905172+0000","last_peered":"2024-09-19T04:04:49.905172+0000","last_clean":"2024-09-19T04:04:49.905172+0000","last_became_active":"2024-09-19T04:04:39.324618+0000","last_became_peered":"2024-09-19T04:04:39.324618+0000","last_unstale":"2024-09-19T04:04:49.905172+0000","last_undegraded":"2024-09-19T04:04:49.905172+0000","last_fullsized":"2024-09-19T04:04:49.905172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:58.748613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351722+0000","last_change":"2024-09-19T04:04:39.318336+0000","last_active":"2024-09-19T04:04:49.351722+0000","last_peered":"2024-09-19T04:04:49.351722+0000","last_clean":"2024-09-19T04:04:49.351722+0000","last_became_active":"2024-09-19T04:04:39.317032+0000","last_became_peered":"2024-09-19T04:04:39.317032+0000","last_unstale":"2024-09-19T04:04:49.351722+0000","last_undegraded":"2024-09-19T04:04:49.351722+0000","last_fullsized":"2024-09-19T04:04:49.351722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:04.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904393+0000","last_change":"2024-09-19T04:04:39.321909+0000","last_active":"2024-09-19T04:04:49.904393+0000","last_peered":"2024-09-19T04:04:49.904393+0000","last_clean":"2024-09-19T04:04:49.904393+0000","last_became_active":"2024-09-19T04:04:39.320522+0000","last_became_peered":"2024-09-19T04:04:39.320522+0000","last_unstale":"2024-09-19T04:04:49.904393+0000","last_undegraded":"2024-09-19T04:04:49.904393+0000","last_fullsized":"2024-09-19T04:04:49.904393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:57.270998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905574+0000","last_change":"2024-09-19T04:04:39.324610+0000","last_active":"2024-09-19T04:04:49.905574+0000","last_peered":"2024-09-19T04:04:49.905574+0000","last_clean":"2024-09-19T04:04:49.905574+0000","last_became_active":"2024-09-19T04:04:39.324190+0000","last_became_peered":"2024-09-19T04:04:39.324190+0000","last_unstale":"2024-09-19T04:04:49.905574+0000","last_undegraded":"2024-09-19T04:04:49.905574+0000","last_fullsized":"2024-09-19T04:04:49.905574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:51.418210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905278+0000","last_change":"2024-09-19T04:04:39.324220+0000","last_active":"2024-09-19T04:04:49.905278+0000","last_peered":"2024-09-19T04:04:49.905278+0000","last_clean":"2024-09-19T04:04:49.905278+0000","last_became_active":"2024-09-19T04:04:39.323890+0000","last_became_peered":"2024-09-19T04:04:39.323890+0000","last_unstale":"2024-09-19T04:04:49.905278+0000","last_undegraded":"2024-09-19T04:04:49.905278+0000","last_fullsized":"2024-09-19T04:04:49.905278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:15.475088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905779+0000","last_change":"2024-09-19T04:04:39.317240+0000","last_active":"2024-09-19T04:04:49.905779+0000","last_peered":"2024-09-19T04:04:49.905779+0000","last_clean":"2024-09-19T04:04:49.905779+0000","last_became_active":"2024-09-19T04:04:39.316740+0000","last_became_peered":"2024-09-19T04:04:39.316740+0000","last_unstale":"2024-09-19T04:04:49.905779+0000","last_undegraded":"2024-09-19T04:04:49.905779+0000","last_fullsized":"2024-09-19T04:04:49.905779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:46.705467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905586+0000","last_change":"2024-09-19T04:04:39.319665+0000","last_active":"2024-09-19T04:04:49.905586+0000","last_peered":"2024-09-19T04:04:49.905586+0000","last_clean":"2024-09-19T04:04:49.905586+0000","last_became_active":"2024-09-19T04:04:39.319227+0000","last_became_peered":"2024-09-19T04:04:39.319227+0000","last_unstale":"2024-09-19T04:04:49.905586+0000","last_undegraded":"2024-09-19T04:04:49.905586+0000","last_fullsized":"2024-09-19T04:04:49.905586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:45.581254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50160,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":155,"num_read_kb":134,"num_write":231,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":150480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248101,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":491318,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710748,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1121479,"data_compressed":53410,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874274,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":495344,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":491626,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":1088125,"data_compressed":53410,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":487145,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1083314,"data_compressed":53410,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":490826,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":486460,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":531347,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":531520,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":488208,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":45317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:05:38.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:05:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:38.577 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:38 smithi019 ceph-mon[30938]: pgmap v317: 129 pgs: 129 active+clean; 626 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 17 op/s 2024-09-19T04:05:38.577 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:38 smithi019 ceph-mon[30938]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:05:38.577 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:38 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:38 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:38 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:05:38.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:38 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:05:38.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:38 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:38 smithi111 ceph-mon[35365]: pgmap v317: 129 pgs: 129 active+clean; 626 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 17 op/s 2024-09-19T04:05:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:38 smithi111 ceph-mon[35365]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:05:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:38 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:38 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:38 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:05:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:38 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:05:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:38 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:38 smithi196 ceph-mon[35731]: pgmap v317: 129 pgs: 129 active+clean; 626 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 17 op/s 2024-09-19T04:05:38.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:38 smithi196 ceph-mon[35731]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:05:38.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:38 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:38 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:38 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:05:38.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:38 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:05:38.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:38 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:05:38.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3766029500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8101de0 msgr2=0x7f5bc81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:38.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3766029500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8101de0 0x7f5bc81021c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009900 tx=0x7f5bbc02a410 comp rx=0 tx=0).stop 2024-09-19T04:05:38.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3766029500 shutdown_connections 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3766029500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bc8106f10 0x7f5bc810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3766029500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bc8102790 0x7f5bc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3766029500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8101de0 0x7f5bc81021c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3766029500 >> 172.21.15.19:0/3766029500 conn(0x7f5bc80fd8c0 msgr2=0x7f5bc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3766029500 shutdown_connections 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.811+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3766029500 wait complete. 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcf1dd640 1 Processor -- start 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcf1dd640 1 -- start start 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcf1dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bc8101de0 0x7f5bc81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcf1dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8102790 0x7f5bc81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcf1dd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bc8106f10 0x7f5bc819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcf1dd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111f70 con 0x7f5bc8102790 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcf1dd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111df0 con 0x7f5bc8106f10 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcf1dd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81120f0 con 0x7f5bc8101de0 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8102790 0x7f5bc81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8102790 0x7f5bc81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56318/0 (socket says 172.21.15.19:56318) 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bc7fff640 1 -- 172.21.15.19:0/3026706521 learned_addr learned my addr 172.21.15.19:0/3026706521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bcd77a640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bc8106f10 0x7f5bc819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.812+0000 7f5bccf79640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bc8101de0 0x7f5bc81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc7fff640 1 -- 172.21.15.19:0/3026706521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bc8101de0 msgr2=0x7f5bc81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc7fff640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bc8101de0 0x7f5bc81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc7fff640 1 -- 172.21.15.19:0/3026706521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bc8106f10 msgr2=0x7f5bc819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc7fff640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bc8106f10 0x7f5bc819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc7fff640 1 -- 172.21.15.19:0/3026706521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc819d460 con 0x7f5bc8102790 2024-09-19T04:05:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc7fff640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8102790 0x7f5bc81989f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400c970 tx=0x7f5bb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc5ffb640 1 -- 172.21.15.19:0/3026706521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4007bb0 con 0x7f5bc8102790 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc5ffb640 1 -- 172.21.15.19:0/3026706521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb4007d50 con 0x7f5bc8102790 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc819d750 con 0x7f5bc8102790 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bc5ffb640 1 -- 172.21.15.19:0/3026706521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400f450 con 0x7f5bc8102790 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.813+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a5030 con 0x7f5bc8102790 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.814+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc81021c0 con 0x7f5bc8102790 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.815+0000 7f5bc5ffb640 1 -- 172.21.15.19:0/3026706521 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5bb4005020 con 0x7f5bc8102790 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.816+0000 7f5bc5ffb640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5bb80a32d0 0x7f5bb80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.816+0000 7f5bccf79640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5bb80a32d0 0x7f5bb80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.816+0000 7f5bc5ffb640 1 -- 172.21.15.19:0/3026706521 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5bb40cfa10 con 0x7f5bc8102790 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.816+0000 7f5bccf79640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5bb80a32d0 0x7f5bb80a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009230 tx=0x7f5bbc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.820+0000 7f5bc5ffb640 1 -- 172.21.15.19:0/3026706521 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5bb4014030 con 0x7f5bc8102790 2024-09-19T04:05:38.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:38.997+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bc810a120 con 0x7f5bb80a32d0 2024-09-19T04:05:39.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.005+0000 7f5bc5ffb640 1 -- 172.21.15.19:0/3026706521 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340639 (secure 0 0 0) 0x7f5bc810a120 con 0x7f5bb80a32d0 2024-09-19T04:05:39.012 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-19T04:05:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5bb80a32d0 msgr2=0x7f5bb80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5bb80a32d0 0x7f5bb80a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009230 tx=0x7f5bbc0058d0 comp rx=0 tx=0).stop 2024-09-19T04:05:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8102790 msgr2=0x7f5bc81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8102790 0x7f5bc81989f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400c970 tx=0x7f5bb400ce30 comp rx=0 tx=0).stop 2024-09-19T04:05:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 shutdown_connections 2024-09-19T04:05:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5bb80a32d0 0x7f5bb80a5790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bc8106f10 0x7f5bc819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8102790 0x7f5bc81989f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 --2- 172.21.15.19:0/3026706521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bc8101de0 0x7f5bc81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 >> 172.21.15.19:0/3026706521 conn(0x7f5bc80fd8c0 msgr2=0x7f5bc80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 shutdown_connections 2024-09-19T04:05:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:39.009+0000 7f5bcf1dd640 1 -- 172.21.15.19:0/3026706521 wait complete. 2024-09-19T04:05:39.714 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-19T04:05:38.422772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654090,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":426,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348800,"kb_used_data":12032,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593024,"statfs":{"total":1151940427776,"available":1151583256576,"internally_reserved":0,"allocated":12320768,"data_stored":7813368,"data_compressed":317721,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53735,"num_objects":9,"num_object_clones":0,"num_object_copies":29,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":143,"num_read_kb":133,"num_write":48,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003614"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:36.627338+0000","last_change":"2024-09-19T04:04:37.214769+0000","last_active":"2024-09-19T04:05:36.627338+0000","last_peered":"2024-09-19T04:05:36.627338+0000","last_clean":"2024-09-19T04:05:36.627338+0000","last_became_active":"2024-09-19T04:04:37.214518+0000","last_became_peered":"2024-09-19T04:04:37.214518+0000","last_unstale":"2024-09-19T04:05:36.627338+0000","last_undegraded":"2024-09-19T04:05:36.627338+0000","last_fullsized":"2024-09-19T04:05:36.627338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:50.704566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885289+0000","last_change":"2024-09-19T04:04:39.329733+0000","last_active":"2024-09-19T04:04:49.885289+0000","last_peered":"2024-09-19T04:04:49.885289+0000","last_clean":"2024-09-19T04:04:49.885289+0000","last_became_active":"2024-09-19T04:04:39.329031+0000","last_became_peered":"2024-09-19T04:04:39.329031+0000","last_unstale":"2024-09-19T04:04:49.885289+0000","last_undegraded":"2024-09-19T04:04:49.885289+0000","last_fullsized":"2024-09-19T04:04:49.885289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:57.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351473+0000","last_change":"2024-09-19T04:04:46.362679+0000","last_active":"2024-09-19T04:04:49.351473+0000","last_peered":"2024-09-19T04:04:49.351473+0000","last_clean":"2024-09-19T04:04:49.351473+0000","last_became_active":"2024-09-19T04:04:46.362257+0000","last_became_peered":"2024-09-19T04:04:46.362257+0000","last_unstale":"2024-09-19T04:04:49.351473+0000","last_undegraded":"2024-09-19T04:04:49.351473+0000","last_fullsized":"2024-09-19T04:04:49.351473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.389839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885530+0000","last_change":"2024-09-19T04:04:37.647295+0000","last_active":"2024-09-19T04:04:49.885530+0000","last_peered":"2024-09-19T04:04:49.885530+0000","last_clean":"2024-09-19T04:04:49.885530+0000","last_became_active":"2024-09-19T04:04:37.646964+0000","last_became_peered":"2024-09-19T04:04:37.646964+0000","last_unstale":"2024-09-19T04:04:49.885530+0000","last_undegraded":"2024-09-19T04:04:49.885530+0000","last_fullsized":"2024-09-19T04:04:49.885530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:49.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905863+0000","last_change":"2024-09-19T04:04:39.318270+0000","last_active":"2024-09-19T04:04:49.905863+0000","last_peered":"2024-09-19T04:04:49.905863+0000","last_clean":"2024-09-19T04:04:49.905863+0000","last_became_active":"2024-09-19T04:04:39.317972+0000","last_became_peered":"2024-09-19T04:04:39.317972+0000","last_unstale":"2024-09-19T04:04:49.905863+0000","last_undegraded":"2024-09-19T04:04:49.905863+0000","last_fullsized":"2024-09-19T04:04:49.905863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:25.994112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904693+0000","last_change":"2024-09-19T04:04:46.355111+0000","last_active":"2024-09-19T04:04:49.904693+0000","last_peered":"2024-09-19T04:04:49.904693+0000","last_clean":"2024-09-19T04:04:49.904693+0000","last_became_active":"2024-09-19T04:04:46.354684+0000","last_became_peered":"2024-09-19T04:04:46.354684+0000","last_unstale":"2024-09-19T04:04:49.904693+0000","last_undegraded":"2024-09-19T04:04:49.904693+0000","last_fullsized":"2024-09-19T04:04:49.904693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:33.141690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.606646+0000","last_change":"2024-09-19T04:04:37.211772+0000","last_active":"2024-09-19T04:05:22.606646+0000","last_peered":"2024-09-19T04:05:22.606646+0000","last_clean":"2024-09-19T04:05:22.606646+0000","last_became_active":"2024-09-19T04:04:37.211452+0000","last_became_peered":"2024-09-19T04:04:37.211452+0000","last_unstale":"2024-09-19T04:05:22.606646+0000","last_undegraded":"2024-09-19T04:05:22.606646+0000","last_fullsized":"2024-09-19T04:05:22.606646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:57.781481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905675+0000","last_change":"2024-09-19T04:04:39.320791+0000","last_active":"2024-09-19T04:04:49.905675+0000","last_peered":"2024-09-19T04:04:49.905675+0000","last_clean":"2024-09-19T04:04:49.905675+0000","last_became_active":"2024-09-19T04:04:39.320206+0000","last_became_peered":"2024-09-19T04:04:39.320206+0000","last_unstale":"2024-09-19T04:04:49.905675+0000","last_undegraded":"2024-09-19T04:04:49.905675+0000","last_fullsized":"2024-09-19T04:04:49.905675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:17.385974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905406+0000","last_change":"2024-09-19T04:04:46.363375+0000","last_active":"2024-09-19T04:04:49.905406+0000","last_peered":"2024-09-19T04:04:49.905406+0000","last_clean":"2024-09-19T04:04:49.905406+0000","last_became_active":"2024-09-19T04:04:46.362800+0000","last_became_peered":"2024-09-19T04:04:46.362800+0000","last_unstale":"2024-09-19T04:04:49.905406+0000","last_undegraded":"2024-09-19T04:04:49.905406+0000","last_fullsized":"2024-09-19T04:04:49.905406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:55.651507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529934+0000","last_change":"2024-09-19T04:04:37.206218+0000","last_active":"2024-09-19T04:05:22.529934+0000","last_peered":"2024-09-19T04:05:22.529934+0000","last_clean":"2024-09-19T04:05:22.529934+0000","last_became_active":"2024-09-19T04:04:37.205675+0000","last_became_peered":"2024-09-19T04:04:37.205675+0000","last_unstale":"2024-09-19T04:05:22.529934+0000","last_undegraded":"2024-09-19T04:05:22.529934+0000","last_fullsized":"2024-09-19T04:05:22.529934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:27.664960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906012+0000","last_change":"2024-09-19T04:04:39.329667+0000","last_active":"2024-09-19T04:04:49.906012+0000","last_peered":"2024-09-19T04:04:49.906012+0000","last_clean":"2024-09-19T04:04:49.906012+0000","last_became_active":"2024-09-19T04:04:39.328180+0000","last_became_peered":"2024-09-19T04:04:39.328180+0000","last_unstale":"2024-09-19T04:04:49.906012+0000","last_undegraded":"2024-09-19T04:04:49.906012+0000","last_fullsized":"2024-09-19T04:04:49.906012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:59.782472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905190+0000","last_change":"2024-09-19T04:04:46.359432+0000","last_active":"2024-09-19T04:04:49.905190+0000","last_peered":"2024-09-19T04:04:49.905190+0000","last_clean":"2024-09-19T04:04:49.905190+0000","last_became_active":"2024-09-19T04:04:46.358966+0000","last_became_peered":"2024-09-19T04:04:46.358966+0000","last_unstale":"2024-09-19T04:04:49.905190+0000","last_undegraded":"2024-09-19T04:04:49.905190+0000","last_fullsized":"2024-09-19T04:04:49.905190+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:41.537926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905315+0000","last_change":"2024-09-19T04:04:39.325084+0000","last_active":"2024-09-19T04:04:49.905315+0000","last_peered":"2024-09-19T04:04:49.905315+0000","last_clean":"2024-09-19T04:04:49.905315+0000","last_became_active":"2024-09-19T04:04:39.324126+0000","last_became_peered":"2024-09-19T04:04:39.324126+0000","last_unstale":"2024-09-19T04:04:49.905315+0000","last_undegraded":"2024-09-19T04:04:49.905315+0000","last_fullsized":"2024-09-19T04:04:49.905315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:13.523889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:19.503198+0000","last_change":"2024-09-19T04:04:37.211548+0000","last_active":"2024-09-19T04:05:19.503198+0000","last_peered":"2024-09-19T04:05:19.503198+0000","last_clean":"2024-09-19T04:05:19.503198+0000","last_became_active":"2024-09-19T04:04:37.211232+0000","last_became_peered":"2024-09-19T04:04:37.211232+0000","last_unstale":"2024-09-19T04:05:19.503198+0000","last_undegraded":"2024-09-19T04:05:19.503198+0000","last_fullsized":"2024-09-19T04:05:19.503198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:35:06.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905215+0000","last_change":"2024-09-19T04:04:46.374921+0000","last_active":"2024-09-19T04:04:49.905215+0000","last_peered":"2024-09-19T04:04:49.905215+0000","last_clean":"2024-09-19T04:04:49.905215+0000","last_became_active":"2024-09-19T04:04:46.374466+0000","last_became_peered":"2024-09-19T04:04:46.374466+0000","last_unstale":"2024-09-19T04:04:49.905215+0000","last_undegraded":"2024-09-19T04:04:49.905215+0000","last_fullsized":"2024-09-19T04:04:49.905215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:46.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884937+0000","last_change":"2024-09-19T04:04:39.319948+0000","last_active":"2024-09-19T04:04:49.884937+0000","last_peered":"2024-09-19T04:04:49.884937+0000","last_clean":"2024-09-19T04:04:49.884937+0000","last_became_active":"2024-09-19T04:04:39.319675+0000","last_became_peered":"2024-09-19T04:04:39.319675+0000","last_unstale":"2024-09-19T04:04:49.884937+0000","last_undegraded":"2024-09-19T04:04:49.884937+0000","last_fullsized":"2024-09-19T04:04:49.884937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:08.818133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:21.514500+0000","last_change":"2024-09-19T04:04:37.207705+0000","last_active":"2024-09-19T04:05:21.514500+0000","last_peered":"2024-09-19T04:05:21.514500+0000","last_clean":"2024-09-19T04:05:21.514500+0000","last_became_active":"2024-09-19T04:04:37.207251+0000","last_became_peered":"2024-09-19T04:04:37.207251+0000","last_unstale":"2024-09-19T04:05:21.514500+0000","last_undegraded":"2024-09-19T04:05:21.514500+0000","last_fullsized":"2024-09-19T04:05:21.514500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:20.835004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:50.337968+0000","last_change":"2024-09-19T04:04:46.354463+0000","last_active":"2024-09-19T04:04:50.337968+0000","last_peered":"2024-09-19T04:04:50.337968+0000","last_clean":"2024-09-19T04:04:50.337968+0000","last_became_active":"2024-09-19T04:04:46.353754+0000","last_became_peered":"2024-09-19T04:04:46.353754+0000","last_unstale":"2024-09-19T04:04:50.337968+0000","last_undegraded":"2024-09-19T04:04:50.337968+0000","last_fullsized":"2024-09-19T04:04:50.337968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:09.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904504+0000","last_change":"2024-09-19T04:04:39.322727+0000","last_active":"2024-09-19T04:04:49.904504+0000","last_peered":"2024-09-19T04:04:49.904504+0000","last_clean":"2024-09-19T04:04:49.904504+0000","last_became_active":"2024-09-19T04:04:39.320989+0000","last_became_peered":"2024-09-19T04:04:39.320989+0000","last_unstale":"2024-09-19T04:04:49.904504+0000","last_undegraded":"2024-09-19T04:04:49.904504+0000","last_fullsized":"2024-09-19T04:04:49.904504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:06.536540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529961+0000","last_change":"2024-09-19T04:04:37.213602+0000","last_active":"2024-09-19T04:05:22.529961+0000","last_peered":"2024-09-19T04:05:22.529961+0000","last_clean":"2024-09-19T04:05:22.529961+0000","last_became_active":"2024-09-19T04:04:37.213240+0000","last_became_peered":"2024-09-19T04:04:37.213240+0000","last_unstale":"2024-09-19T04:05:22.529961+0000","last_undegraded":"2024-09-19T04:05:22.529961+0000","last_fullsized":"2024-09-19T04:05:22.529961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:09.822393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350925+0000","last_change":"2024-09-19T04:04:46.363921+0000","last_active":"2024-09-19T04:04:49.350925+0000","last_peered":"2024-09-19T04:04:49.350925+0000","last_clean":"2024-09-19T04:04:49.350925+0000","last_became_active":"2024-09-19T04:04:46.363476+0000","last_became_peered":"2024-09-19T04:04:46.363476+0000","last_unstale":"2024-09-19T04:04:49.350925+0000","last_undegraded":"2024-09-19T04:04:49.350925+0000","last_fullsized":"2024-09-19T04:04:49.350925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:32.015782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904206+0000","last_change":"2024-09-19T04:04:39.316814+0000","last_active":"2024-09-19T04:04:49.904206+0000","last_peered":"2024-09-19T04:04:49.904206+0000","last_clean":"2024-09-19T04:04:49.904206+0000","last_became_active":"2024-09-19T04:04:39.316504+0000","last_became_peered":"2024-09-19T04:04:39.316504+0000","last_unstale":"2024-09-19T04:04:49.904206+0000","last_undegraded":"2024-09-19T04:04:49.904206+0000","last_fullsized":"2024-09-19T04:04:49.904206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:19.503333+0000","last_change":"2024-09-19T04:04:37.647260+0000","last_active":"2024-09-19T04:05:19.503333+0000","last_peered":"2024-09-19T04:05:19.503333+0000","last_clean":"2024-09-19T04:05:19.503333+0000","last_became_active":"2024-09-19T04:04:37.646971+0000","last_became_peered":"2024-09-19T04:04:37.646971+0000","last_unstale":"2024-09-19T04:05:19.503333+0000","last_undegraded":"2024-09-19T04:05:19.503333+0000","last_fullsized":"2024-09-19T04:05:19.503333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:07.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905176+0000","last_change":"2024-09-19T04:04:46.375999+0000","last_active":"2024-09-19T04:04:49.905176+0000","last_peered":"2024-09-19T04:04:49.905176+0000","last_clean":"2024-09-19T04:04:49.905176+0000","last_became_active":"2024-09-19T04:04:46.375602+0000","last_became_peered":"2024-09-19T04:04:46.375602+0000","last_unstale":"2024-09-19T04:04:49.905176+0000","last_undegraded":"2024-09-19T04:04:49.905176+0000","last_fullsized":"2024-09-19T04:04:49.905176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:18.952629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905926+0000","last_change":"2024-09-19T04:04:39.318185+0000","last_active":"2024-09-19T04:04:49.905926+0000","last_peered":"2024-09-19T04:04:49.905926+0000","last_clean":"2024-09-19T04:04:49.905926+0000","last_became_active":"2024-09-19T04:04:39.317885+0000","last_became_peered":"2024-09-19T04:04:39.317885+0000","last_unstale":"2024-09-19T04:04:49.905926+0000","last_undegraded":"2024-09-19T04:04:49.905926+0000","last_fullsized":"2024-09-19T04:04:49.905926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:27.719248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529369+0000","last_change":"2024-09-19T04:04:37.703740+0000","last_active":"2024-09-19T04:05:22.529369+0000","last_peered":"2024-09-19T04:05:22.529369+0000","last_clean":"2024-09-19T04:05:22.529369+0000","last_became_active":"2024-09-19T04:04:37.703414+0000","last_became_peered":"2024-09-19T04:04:37.703414+0000","last_unstale":"2024-09-19T04:05:22.529369+0000","last_undegraded":"2024-09-19T04:05:22.529369+0000","last_fullsized":"2024-09-19T04:05:22.529369+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:19.653559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351586+0000","last_change":"2024-09-19T04:04:46.354964+0000","last_active":"2024-09-19T04:04:49.351586+0000","last_peered":"2024-09-19T04:04:49.351586+0000","last_clean":"2024-09-19T04:04:49.351586+0000","last_became_active":"2024-09-19T04:04:46.354494+0000","last_became_peered":"2024-09-19T04:04:46.354494+0000","last_unstale":"2024-09-19T04:04:49.351586+0000","last_undegraded":"2024-09-19T04:04:49.351586+0000","last_fullsized":"2024-09-19T04:04:49.351586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:11.246134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885406+0000","last_change":"2024-09-19T04:04:39.315811+0000","last_active":"2024-09-19T04:04:49.885406+0000","last_peered":"2024-09-19T04:04:49.885406+0000","last_clean":"2024-09-19T04:04:49.885406+0000","last_became_active":"2024-09-19T04:04:39.315466+0000","last_became_peered":"2024-09-19T04:04:39.315466+0000","last_unstale":"2024-09-19T04:04:49.885406+0000","last_undegraded":"2024-09-19T04:04:49.885406+0000","last_fullsized":"2024-09-19T04:04:49.885406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:17.414765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'15","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:36.317616+0000","last_change":"2024-09-19T04:04:37.204206+0000","last_active":"2024-09-19T04:05:36.317616+0000","last_peered":"2024-09-19T04:05:36.317616+0000","last_clean":"2024-09-19T04:05:36.317616+0000","last_became_active":"2024-09-19T04:04:37.203845+0000","last_became_peered":"2024-09-19T04:04:37.203845+0000","last_unstale":"2024-09-19T04:05:36.317616+0000","last_undegraded":"2024-09-19T04:05:36.317616+0000","last_fullsized":"2024-09-19T04:05:36.317616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:52.868643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884707+0000","last_change":"2024-09-19T04:04:46.375541+0000","last_active":"2024-09-19T04:04:49.884707+0000","last_peered":"2024-09-19T04:04:49.884707+0000","last_clean":"2024-09-19T04:04:49.884707+0000","last_became_active":"2024-09-19T04:04:46.375142+0000","last_became_peered":"2024-09-19T04:04:46.375142+0000","last_unstale":"2024-09-19T04:04:49.884707+0000","last_undegraded":"2024-09-19T04:04:49.884707+0000","last_fullsized":"2024-09-19T04:04:49.884707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.889459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904952+0000","last_change":"2024-09-19T04:04:39.324094+0000","last_active":"2024-09-19T04:04:49.904952+0000","last_peered":"2024-09-19T04:04:49.904952+0000","last_clean":"2024-09-19T04:04:49.904952+0000","last_became_active":"2024-09-19T04:04:39.322897+0000","last_became_peered":"2024-09-19T04:04:39.322897+0000","last_unstale":"2024-09-19T04:04:49.904952+0000","last_undegraded":"2024-09-19T04:04:49.904952+0000","last_fullsized":"2024-09-19T04:04:49.904952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:07.773110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529315+0000","last_change":"2024-09-19T04:04:37.647686+0000","last_active":"2024-09-19T04:05:22.529315+0000","last_peered":"2024-09-19T04:05:22.529315+0000","last_clean":"2024-09-19T04:05:22.529315+0000","last_became_active":"2024-09-19T04:04:37.646979+0000","last_became_peered":"2024-09-19T04:04:37.646979+0000","last_unstale":"2024-09-19T04:05:22.529315+0000","last_undegraded":"2024-09-19T04:05:22.529315+0000","last_fullsized":"2024-09-19T04:05:22.529315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:48.063075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905896+0000","last_change":"2024-09-19T04:04:46.356078+0000","last_active":"2024-09-19T04:04:49.905896+0000","last_peered":"2024-09-19T04:04:49.905896+0000","last_clean":"2024-09-19T04:04:49.905896+0000","last_became_active":"2024-09-19T04:04:46.355630+0000","last_became_peered":"2024-09-19T04:04:46.355630+0000","last_unstale":"2024-09-19T04:04:49.905896+0000","last_undegraded":"2024-09-19T04:04:49.905896+0000","last_fullsized":"2024-09-19T04:04:49.905896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:45.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351620+0000","last_change":"2024-09-19T04:04:39.311067+0000","last_active":"2024-09-19T04:04:49.351620+0000","last_peered":"2024-09-19T04:04:49.351620+0000","last_clean":"2024-09-19T04:04:49.351620+0000","last_became_active":"2024-09-19T04:04:39.310725+0000","last_became_peered":"2024-09-19T04:04:39.310725+0000","last_unstale":"2024-09-19T04:04:49.351620+0000","last_undegraded":"2024-09-19T04:04:49.351620+0000","last_fullsized":"2024-09-19T04:04:49.351620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:47:11.488121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351028+0000","last_change":"2024-09-19T04:04:37.648335+0000","last_active":"2024-09-19T04:04:49.351028+0000","last_peered":"2024-09-19T04:04:49.351028+0000","last_clean":"2024-09-19T04:04:49.351028+0000","last_became_active":"2024-09-19T04:04:37.648000+0000","last_became_peered":"2024-09-19T04:04:37.648000+0000","last_unstale":"2024-09-19T04:04:49.351028+0000","last_undegraded":"2024-09-19T04:04:49.351028+0000","last_fullsized":"2024-09-19T04:04:49.351028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:16.102728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904470+0000","last_change":"2024-09-19T04:04:46.375653+0000","last_active":"2024-09-19T04:04:49.904470+0000","last_peered":"2024-09-19T04:04:49.904470+0000","last_clean":"2024-09-19T04:04:49.904470+0000","last_became_active":"2024-09-19T04:04:46.375132+0000","last_became_peered":"2024-09-19T04:04:46.375132+0000","last_unstale":"2024-09-19T04:04:49.904470+0000","last_undegraded":"2024-09-19T04:04:49.904470+0000","last_fullsized":"2024-09-19T04:04:49.904470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:03.332697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351643+0000","last_change":"2024-09-19T04:04:39.318155+0000","last_active":"2024-09-19T04:04:49.351643+0000","last_peered":"2024-09-19T04:04:49.351643+0000","last_clean":"2024-09-19T04:04:49.351643+0000","last_became_active":"2024-09-19T04:04:39.316710+0000","last_became_peered":"2024-09-19T04:04:39.316710+0000","last_unstale":"2024-09-19T04:04:49.351643+0000","last_undegraded":"2024-09-19T04:04:49.351643+0000","last_fullsized":"2024-09-19T04:04:49.351643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:59.084523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:33.624368+0000","last_change":"2024-09-19T04:04:37.210695+0000","last_active":"2024-09-19T04:05:33.624368+0000","last_peered":"2024-09-19T04:05:33.624368+0000","last_clean":"2024-09-19T04:05:33.624368+0000","last_became_active":"2024-09-19T04:04:37.210334+0000","last_became_peered":"2024-09-19T04:04:37.210334+0000","last_unstale":"2024-09-19T04:05:33.624368+0000","last_undegraded":"2024-09-19T04:05:33.624368+0000","last_fullsized":"2024-09-19T04:05:33.624368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:12.545248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904818+0000","last_change":"2024-09-19T04:04:46.358130+0000","last_active":"2024-09-19T04:04:49.904818+0000","last_peered":"2024-09-19T04:04:49.904818+0000","last_clean":"2024-09-19T04:04:49.904818+0000","last_became_active":"2024-09-19T04:04:46.356810+0000","last_became_peered":"2024-09-19T04:04:46.356810+0000","last_unstale":"2024-09-19T04:04:49.904818+0000","last_undegraded":"2024-09-19T04:04:49.904818+0000","last_fullsized":"2024-09-19T04:04:49.904818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:49.884478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905394+0000","last_change":"2024-09-19T04:04:39.325866+0000","last_active":"2024-09-19T04:04:49.905394+0000","last_peered":"2024-09-19T04:04:49.905394+0000","last_clean":"2024-09-19T04:04:49.905394+0000","last_became_active":"2024-09-19T04:04:39.325473+0000","last_became_peered":"2024-09-19T04:04:39.325473+0000","last_unstale":"2024-09-19T04:04:49.905394+0000","last_undegraded":"2024-09-19T04:04:49.905394+0000","last_fullsized":"2024-09-19T04:04:49.905394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:30.036704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:34.625947+0000","last_change":"2024-09-19T04:04:37.647310+0000","last_active":"2024-09-19T04:05:34.625947+0000","last_peered":"2024-09-19T04:05:34.625947+0000","last_clean":"2024-09-19T04:05:34.625947+0000","last_became_active":"2024-09-19T04:04:37.646903+0000","last_became_peered":"2024-09-19T04:04:37.646903+0000","last_unstale":"2024-09-19T04:05:34.625947+0000","last_undegraded":"2024-09-19T04:05:34.625947+0000","last_fullsized":"2024-09-19T04:05:34.625947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:20.097472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904925+0000","last_change":"2024-09-19T04:04:46.375261+0000","last_active":"2024-09-19T04:04:49.904925+0000","last_peered":"2024-09-19T04:04:49.904925+0000","last_clean":"2024-09-19T04:04:49.904925+0000","last_became_active":"2024-09-19T04:04:46.374489+0000","last_became_peered":"2024-09-19T04:04:46.374489+0000","last_unstale":"2024-09-19T04:04:49.904925+0000","last_undegraded":"2024-09-19T04:04:49.904925+0000","last_fullsized":"2024-09-19T04:04:49.904925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:05.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904224+0000","last_change":"2024-09-19T04:04:39.322265+0000","last_active":"2024-09-19T04:04:49.904224+0000","last_peered":"2024-09-19T04:04:49.904224+0000","last_clean":"2024-09-19T04:04:49.904224+0000","last_became_active":"2024-09-19T04:04:39.321938+0000","last_became_peered":"2024-09-19T04:04:39.321938+0000","last_unstale":"2024-09-19T04:04:49.904224+0000","last_undegraded":"2024-09-19T04:04:49.904224+0000","last_fullsized":"2024-09-19T04:04:49.904224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:19.220661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:20.508304+0000","last_change":"2024-09-19T04:04:37.211330+0000","last_active":"2024-09-19T04:05:20.508304+0000","last_peered":"2024-09-19T04:05:20.508304+0000","last_clean":"2024-09-19T04:05:20.508304+0000","last_became_active":"2024-09-19T04:04:37.210788+0000","last_became_peered":"2024-09-19T04:04:37.210788+0000","last_unstale":"2024-09-19T04:05:20.508304+0000","last_undegraded":"2024-09-19T04:05:20.508304+0000","last_fullsized":"2024-09-19T04:05:20.508304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:43.169897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905589+0000","last_change":"2024-09-19T04:04:46.376030+0000","last_active":"2024-09-19T04:04:49.905589+0000","last_peered":"2024-09-19T04:04:49.905589+0000","last_clean":"2024-09-19T04:04:49.905589+0000","last_became_active":"2024-09-19T04:04:46.375386+0000","last_became_peered":"2024-09-19T04:04:46.375386+0000","last_unstale":"2024-09-19T04:04:49.905589+0000","last_undegraded":"2024-09-19T04:04:49.905589+0000","last_fullsized":"2024-09-19T04:04:49.905589+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:39.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884994+0000","last_change":"2024-09-19T04:04:39.329480+0000","last_active":"2024-09-19T04:04:49.884994+0000","last_peered":"2024-09-19T04:04:49.884994+0000","last_clean":"2024-09-19T04:04:49.884994+0000","last_became_active":"2024-09-19T04:04:39.329008+0000","last_became_peered":"2024-09-19T04:04:39.329008+0000","last_unstale":"2024-09-19T04:04:49.884994+0000","last_undegraded":"2024-09-19T04:04:49.884994+0000","last_fullsized":"2024-09-19T04:04:49.884994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:06.782579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.608678+0000","last_change":"2024-09-19T04:04:37.216224+0000","last_active":"2024-09-19T04:05:22.608678+0000","last_peered":"2024-09-19T04:05:22.608678+0000","last_clean":"2024-09-19T04:05:22.608678+0000","last_became_active":"2024-09-19T04:04:37.215846+0000","last_became_peered":"2024-09-19T04:04:37.215846+0000","last_unstale":"2024-09-19T04:05:22.608678+0000","last_undegraded":"2024-09-19T04:05:22.608678+0000","last_fullsized":"2024-09-19T04:05:22.608678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:57.829694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905099+0000","last_change":"2024-09-19T04:03:25.869634+0000","last_active":"2024-09-19T04:04:49.905099+0000","last_peered":"2024-09-19T04:04:49.905099+0000","last_clean":"2024-09-19T04:04:49.905099+0000","last_became_active":"2024-09-19T04:03:25.869250+0000","last_became_peered":"2024-09-19T04:03:25.869250+0000","last_unstale":"2024-09-19T04:04:49.905099+0000","last_undegraded":"2024-09-19T04:04:49.905099+0000","last_fullsized":"2024-09-19T04:04:49.905099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_clean_scrub_stamp":"2024-09-19T03:59:59.253417+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:51.956932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905834+0000","last_change":"2024-09-19T04:04:46.357123+0000","last_active":"2024-09-19T04:04:49.905834+0000","last_peered":"2024-09-19T04:04:49.905834+0000","last_clean":"2024-09-19T04:04:49.905834+0000","last_became_active":"2024-09-19T04:04:46.356671+0000","last_became_peered":"2024-09-19T04:04:46.356671+0000","last_unstale":"2024-09-19T04:04:49.905834+0000","last_undegraded":"2024-09-19T04:04:49.905834+0000","last_fullsized":"2024-09-19T04:04:49.905834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:09.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905969+0000","last_change":"2024-09-19T04:04:39.322487+0000","last_active":"2024-09-19T04:04:49.905969+0000","last_peered":"2024-09-19T04:04:49.905969+0000","last_clean":"2024-09-19T04:04:49.905969+0000","last_became_active":"2024-09-19T04:04:39.321994+0000","last_became_peered":"2024-09-19T04:04:39.321994+0000","last_unstale":"2024-09-19T04:04:49.905969+0000","last_undegraded":"2024-09-19T04:04:49.905969+0000","last_fullsized":"2024-09-19T04:04:49.905969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:56.977405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:36.628616+0000","last_change":"2024-09-19T04:04:37.703452+0000","last_active":"2024-09-19T04:05:36.628616+0000","last_peered":"2024-09-19T04:05:36.628616+0000","last_clean":"2024-09-19T04:05:36.628616+0000","last_became_active":"2024-09-19T04:04:37.703048+0000","last_became_peered":"2024-09-19T04:04:37.703048+0000","last_unstale":"2024-09-19T04:05:36.628616+0000","last_undegraded":"2024-09-19T04:05:36.628616+0000","last_fullsized":"2024-09-19T04:05:36.628616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:16:29.051617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906348+0000","last_change":"2024-09-19T04:04:46.361229+0000","last_active":"2024-09-19T04:04:49.906348+0000","last_peered":"2024-09-19T04:04:49.906348+0000","last_clean":"2024-09-19T04:04:49.906348+0000","last_became_active":"2024-09-19T04:04:46.360729+0000","last_became_peered":"2024-09-19T04:04:46.360729+0000","last_unstale":"2024-09-19T04:04:49.906348+0000","last_undegraded":"2024-09-19T04:04:49.906348+0000","last_fullsized":"2024-09-19T04:04:49.906348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:48.022778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905796+0000","last_change":"2024-09-19T04:04:39.328598+0000","last_active":"2024-09-19T04:04:49.905796+0000","last_peered":"2024-09-19T04:04:49.905796+0000","last_clean":"2024-09-19T04:04:49.905796+0000","last_became_active":"2024-09-19T04:04:39.328247+0000","last_became_peered":"2024-09-19T04:04:39.328247+0000","last_unstale":"2024-09-19T04:04:49.905796+0000","last_undegraded":"2024-09-19T04:04:49.905796+0000","last_fullsized":"2024-09-19T04:04:49.905796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:34.916487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529955+0000","last_change":"2024-09-19T04:04:37.212065+0000","last_active":"2024-09-19T04:05:22.529955+0000","last_peered":"2024-09-19T04:05:22.529955+0000","last_clean":"2024-09-19T04:05:22.529955+0000","last_became_active":"2024-09-19T04:04:37.211520+0000","last_became_peered":"2024-09-19T04:04:37.211520+0000","last_unstale":"2024-09-19T04:05:22.529955+0000","last_undegraded":"2024-09-19T04:05:22.529955+0000","last_fullsized":"2024-09-19T04:05:22.529955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:53.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905684+0000","last_change":"2024-09-19T04:04:46.358558+0000","last_active":"2024-09-19T04:04:49.905684+0000","last_peered":"2024-09-19T04:04:49.905684+0000","last_clean":"2024-09-19T04:04:49.905684+0000","last_became_active":"2024-09-19T04:04:46.358158+0000","last_became_peered":"2024-09-19T04:04:46.358158+0000","last_unstale":"2024-09-19T04:04:49.905684+0000","last_undegraded":"2024-09-19T04:04:49.905684+0000","last_fullsized":"2024-09-19T04:04:49.905684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:16.627051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906045+0000","last_change":"2024-09-19T04:04:39.325187+0000","last_active":"2024-09-19T04:04:49.906045+0000","last_peered":"2024-09-19T04:04:49.906045+0000","last_clean":"2024-09-19T04:04:49.906045+0000","last_became_active":"2024-09-19T04:04:39.324644+0000","last_became_peered":"2024-09-19T04:04:39.324644+0000","last_unstale":"2024-09-19T04:04:49.906045+0000","last_undegraded":"2024-09-19T04:04:49.906045+0000","last_fullsized":"2024-09-19T04:04:49.906045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:11.879273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.528447+0000","last_change":"2024-09-19T04:04:37.205389+0000","last_active":"2024-09-19T04:05:22.528447+0000","last_peered":"2024-09-19T04:05:22.528447+0000","last_clean":"2024-09-19T04:05:22.528447+0000","last_became_active":"2024-09-19T04:04:37.205117+0000","last_became_peered":"2024-09-19T04:04:37.205117+0000","last_unstale":"2024-09-19T04:05:22.528447+0000","last_undegraded":"2024-09-19T04:05:22.528447+0000","last_fullsized":"2024-09-19T04:05:22.528447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:49.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904546+0000","last_change":"2024-09-19T04:04:46.359815+0000","last_active":"2024-09-19T04:04:49.904546+0000","last_peered":"2024-09-19T04:04:49.904546+0000","last_clean":"2024-09-19T04:04:49.904546+0000","last_became_active":"2024-09-19T04:04:46.359185+0000","last_became_peered":"2024-09-19T04:04:46.359185+0000","last_unstale":"2024-09-19T04:04:49.904546+0000","last_undegraded":"2024-09-19T04:04:49.904546+0000","last_fullsized":"2024-09-19T04:04:49.904546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:54.872722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905210+0000","last_change":"2024-09-19T04:04:39.328841+0000","last_active":"2024-09-19T04:04:49.905210+0000","last_peered":"2024-09-19T04:04:49.905210+0000","last_clean":"2024-09-19T04:04:49.905210+0000","last_became_active":"2024-09-19T04:04:39.328470+0000","last_became_peered":"2024-09-19T04:04:39.328470+0000","last_unstale":"2024-09-19T04:04:49.905210+0000","last_undegraded":"2024-09-19T04:04:49.905210+0000","last_fullsized":"2024-09-19T04:04:49.905210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:17.701469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:21.504227+0000","last_change":"2024-09-19T04:04:37.208116+0000","last_active":"2024-09-19T04:05:21.504227+0000","last_peered":"2024-09-19T04:05:21.504227+0000","last_clean":"2024-09-19T04:05:21.504227+0000","last_became_active":"2024-09-19T04:04:37.207720+0000","last_became_peered":"2024-09-19T04:04:37.207720+0000","last_unstale":"2024-09-19T04:05:21.504227+0000","last_undegraded":"2024-09-19T04:05:21.504227+0000","last_fullsized":"2024-09-19T04:05:21.504227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:01.191545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905195+0000","last_change":"2024-09-19T04:04:46.361199+0000","last_active":"2024-09-19T04:04:49.905195+0000","last_peered":"2024-09-19T04:04:49.905195+0000","last_clean":"2024-09-19T04:04:49.905195+0000","last_became_active":"2024-09-19T04:04:46.360639+0000","last_became_peered":"2024-09-19T04:04:46.360639+0000","last_unstale":"2024-09-19T04:04:49.905195+0000","last_undegraded":"2024-09-19T04:04:49.905195+0000","last_fullsized":"2024-09-19T04:04:49.905195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:50.185760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905226+0000","last_change":"2024-09-19T04:04:39.328529+0000","last_active":"2024-09-19T04:04:49.905226+0000","last_peered":"2024-09-19T04:04:49.905226+0000","last_clean":"2024-09-19T04:04:49.905226+0000","last_became_active":"2024-09-19T04:04:39.328174+0000","last_became_peered":"2024-09-19T04:04:39.328174+0000","last_unstale":"2024-09-19T04:04:49.905226+0000","last_undegraded":"2024-09-19T04:04:49.905226+0000","last_fullsized":"2024-09-19T04:04:49.905226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:02.503672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.530341+0000","last_change":"2024-09-19T04:04:37.647919+0000","last_active":"2024-09-19T04:05:22.530341+0000","last_peered":"2024-09-19T04:05:22.530341+0000","last_clean":"2024-09-19T04:05:22.530341+0000","last_became_active":"2024-09-19T04:04:37.647305+0000","last_became_peered":"2024-09-19T04:04:37.647305+0000","last_unstale":"2024-09-19T04:05:22.530341+0000","last_undegraded":"2024-09-19T04:05:22.530341+0000","last_fullsized":"2024-09-19T04:05:22.530341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:07.035465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905134+0000","last_change":"2024-09-19T04:04:46.360362+0000","last_active":"2024-09-19T04:04:49.905134+0000","last_peered":"2024-09-19T04:04:49.905134+0000","last_clean":"2024-09-19T04:04:49.905134+0000","last_became_active":"2024-09-19T04:04:46.359470+0000","last_became_peered":"2024-09-19T04:04:46.359470+0000","last_unstale":"2024-09-19T04:04:49.905134+0000","last_undegraded":"2024-09-19T04:04:49.905134+0000","last_fullsized":"2024-09-19T04:04:49.905134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:43.804073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904892+0000","last_change":"2024-09-19T04:04:39.326090+0000","last_active":"2024-09-19T04:04:49.904892+0000","last_peered":"2024-09-19T04:04:49.904892+0000","last_clean":"2024-09-19T04:04:49.904892+0000","last_became_active":"2024-09-19T04:04:39.325751+0000","last_became_peered":"2024-09-19T04:04:39.325751+0000","last_unstale":"2024-09-19T04:04:49.904892+0000","last_undegraded":"2024-09-19T04:04:49.904892+0000","last_fullsized":"2024-09-19T04:04:49.904892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:25.357467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:33.625266+0000","last_change":"2024-09-19T04:04:37.215107+0000","last_active":"2024-09-19T04:05:33.625266+0000","last_peered":"2024-09-19T04:05:33.625266+0000","last_clean":"2024-09-19T04:05:33.625266+0000","last_became_active":"2024-09-19T04:04:37.214693+0000","last_became_peered":"2024-09-19T04:04:37.214693+0000","last_unstale":"2024-09-19T04:05:33.625266+0000","last_undegraded":"2024-09-19T04:05:33.625266+0000","last_fullsized":"2024-09-19T04:05:33.625266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:56.535316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350946+0000","last_change":"2024-09-19T04:04:46.363625+0000","last_active":"2024-09-19T04:04:49.350946+0000","last_peered":"2024-09-19T04:04:49.350946+0000","last_clean":"2024-09-19T04:04:49.350946+0000","last_became_active":"2024-09-19T04:04:46.363207+0000","last_became_peered":"2024-09-19T04:04:46.363207+0000","last_unstale":"2024-09-19T04:04:49.350946+0000","last_undegraded":"2024-09-19T04:04:49.350946+0000","last_fullsized":"2024-09-19T04:04:49.350946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:30.251168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905446+0000","last_change":"2024-09-19T04:04:39.317977+0000","last_active":"2024-09-19T04:04:49.905446+0000","last_peered":"2024-09-19T04:04:49.905446+0000","last_clean":"2024-09-19T04:04:49.905446+0000","last_became_active":"2024-09-19T04:04:39.317658+0000","last_became_peered":"2024-09-19T04:04:39.317658+0000","last_unstale":"2024-09-19T04:04:49.905446+0000","last_undegraded":"2024-09-19T04:04:49.905446+0000","last_fullsized":"2024-09-19T04:04:49.905446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:34.370854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529778+0000","last_change":"2024-09-19T04:04:37.212103+0000","last_active":"2024-09-19T04:05:22.529778+0000","last_peered":"2024-09-19T04:05:22.529778+0000","last_clean":"2024-09-19T04:05:22.529778+0000","last_became_active":"2024-09-19T04:04:37.211739+0000","last_became_peered":"2024-09-19T04:04:37.211739+0000","last_unstale":"2024-09-19T04:05:22.529778+0000","last_undegraded":"2024-09-19T04:05:22.529778+0000","last_fullsized":"2024-09-19T04:05:22.529778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:06.588194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904937+0000","last_change":"2024-09-19T04:04:46.361975+0000","last_active":"2024-09-19T04:04:49.904937+0000","last_peered":"2024-09-19T04:04:49.904937+0000","last_clean":"2024-09-19T04:04:49.904937+0000","last_became_active":"2024-09-19T04:04:46.361492+0000","last_became_peered":"2024-09-19T04:04:46.361492+0000","last_unstale":"2024-09-19T04:04:49.904937+0000","last_undegraded":"2024-09-19T04:04:49.904937+0000","last_fullsized":"2024-09-19T04:04:49.904937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:52.723836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904429+0000","last_change":"2024-09-19T04:04:39.317756+0000","last_active":"2024-09-19T04:04:49.904429+0000","last_peered":"2024-09-19T04:04:49.904429+0000","last_clean":"2024-09-19T04:04:49.904429+0000","last_became_active":"2024-09-19T04:04:39.317322+0000","last_became_peered":"2024-09-19T04:04:39.317322+0000","last_unstale":"2024-09-19T04:04:49.904429+0000","last_undegraded":"2024-09-19T04:04:49.904429+0000","last_fullsized":"2024-09-19T04:04:49.904429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:26.291574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.606813+0000","last_change":"2024-09-19T04:04:37.213892+0000","last_active":"2024-09-19T04:05:22.606813+0000","last_peered":"2024-09-19T04:05:22.606813+0000","last_clean":"2024-09-19T04:05:22.606813+0000","last_became_active":"2024-09-19T04:04:37.213350+0000","last_became_peered":"2024-09-19T04:04:37.213350+0000","last_unstale":"2024-09-19T04:05:22.606813+0000","last_undegraded":"2024-09-19T04:05:22.606813+0000","last_fullsized":"2024-09-19T04:05:22.606813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:40.065717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905896+0000","last_change":"2024-09-19T04:04:46.360566+0000","last_active":"2024-09-19T04:04:49.905896+0000","last_peered":"2024-09-19T04:04:49.905896+0000","last_clean":"2024-09-19T04:04:49.905896+0000","last_became_active":"2024-09-19T04:04:46.360143+0000","last_became_peered":"2024-09-19T04:04:46.360143+0000","last_unstale":"2024-09-19T04:04:49.905896+0000","last_undegraded":"2024-09-19T04:04:49.905896+0000","last_fullsized":"2024-09-19T04:04:49.905896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:59.997088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905190+0000","last_change":"2024-09-19T04:04:39.318363+0000","last_active":"2024-09-19T04:04:49.905190+0000","last_peered":"2024-09-19T04:04:49.905190+0000","last_clean":"2024-09-19T04:04:49.905190+0000","last_became_active":"2024-09-19T04:04:39.318003+0000","last_became_peered":"2024-09-19T04:04:39.318003+0000","last_unstale":"2024-09-19T04:04:49.905190+0000","last_undegraded":"2024-09-19T04:04:49.905190+0000","last_fullsized":"2024-09-19T04:04:49.905190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:54.128795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'31","reported_seq":99,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:37.632672+0000","last_change":"2024-09-19T04:04:37.212614+0000","last_active":"2024-09-19T04:05:37.632672+0000","last_peered":"2024-09-19T04:05:37.632672+0000","last_clean":"2024-09-19T04:05:37.632672+0000","last_became_active":"2024-09-19T04:04:37.212153+0000","last_became_peered":"2024-09-19T04:04:37.212153+0000","last_unstale":"2024-09-19T04:05:37.632672+0000","last_undegraded":"2024-09-19T04:05:37.632672+0000","last_fullsized":"2024-09-19T04:05:37.632672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:11.930287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56673,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":75,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350608+0000","last_change":"2024-09-19T04:04:46.362967+0000","last_active":"2024-09-19T04:04:49.350608+0000","last_peered":"2024-09-19T04:04:49.350608+0000","last_clean":"2024-09-19T04:04:49.350608+0000","last_became_active":"2024-09-19T04:04:46.362529+0000","last_became_peered":"2024-09-19T04:04:46.362529+0000","last_unstale":"2024-09-19T04:04:49.350608+0000","last_undegraded":"2024-09-19T04:04:49.350608+0000","last_fullsized":"2024-09-19T04:04:49.350608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:01.821697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905771+0000","last_change":"2024-09-19T04:04:39.323499+0000","last_active":"2024-09-19T04:04:49.905771+0000","last_peered":"2024-09-19T04:04:49.905771+0000","last_clean":"2024-09-19T04:04:49.905771+0000","last_became_active":"2024-09-19T04:04:39.323076+0000","last_became_peered":"2024-09-19T04:04:39.323076+0000","last_unstale":"2024-09-19T04:04:49.905771+0000","last_undegraded":"2024-09-19T04:04:49.905771+0000","last_fullsized":"2024-09-19T04:04:49.905771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:10.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:21.513796+0000","last_change":"2024-09-19T04:04:37.647927+0000","last_active":"2024-09-19T04:05:21.513796+0000","last_peered":"2024-09-19T04:05:21.513796+0000","last_clean":"2024-09-19T04:05:21.513796+0000","last_became_active":"2024-09-19T04:04:37.647279+0000","last_became_peered":"2024-09-19T04:04:37.647279+0000","last_unstale":"2024-09-19T04:05:21.513796+0000","last_undegraded":"2024-09-19T04:05:21.513796+0000","last_fullsized":"2024-09-19T04:05:21.513796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:05:50.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904874+0000","last_change":"2024-09-19T04:04:46.363562+0000","last_active":"2024-09-19T04:04:49.904874+0000","last_peered":"2024-09-19T04:04:49.904874+0000","last_clean":"2024-09-19T04:04:49.904874+0000","last_became_active":"2024-09-19T04:04:46.362915+0000","last_became_peered":"2024-09-19T04:04:46.362915+0000","last_unstale":"2024-09-19T04:04:49.904874+0000","last_undegraded":"2024-09-19T04:04:49.904874+0000","last_fullsized":"2024-09-19T04:04:49.904874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:08.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905947+0000","last_change":"2024-09-19T04:04:39.329034+0000","last_active":"2024-09-19T04:04:49.905947+0000","last_peered":"2024-09-19T04:04:49.905947+0000","last_clean":"2024-09-19T04:04:49.905947+0000","last_became_active":"2024-09-19T04:04:39.328692+0000","last_became_peered":"2024-09-19T04:04:39.328692+0000","last_unstale":"2024-09-19T04:04:49.905947+0000","last_undegraded":"2024-09-19T04:04:49.905947+0000","last_fullsized":"2024-09-19T04:04:49.905947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:42.903089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.529488+0000","last_change":"2024-09-19T04:04:37.212393+0000","last_active":"2024-09-19T04:05:22.529488+0000","last_peered":"2024-09-19T04:05:22.529488+0000","last_clean":"2024-09-19T04:05:22.529488+0000","last_became_active":"2024-09-19T04:04:37.212013+0000","last_became_peered":"2024-09-19T04:04:37.212013+0000","last_unstale":"2024-09-19T04:05:22.529488+0000","last_undegraded":"2024-09-19T04:05:22.529488+0000","last_fullsized":"2024-09-19T04:05:22.529488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:37.981075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350389+0000","last_change":"2024-09-19T04:04:46.359239+0000","last_active":"2024-09-19T04:04:49.350389+0000","last_peered":"2024-09-19T04:04:49.350389+0000","last_clean":"2024-09-19T04:04:49.350389+0000","last_became_active":"2024-09-19T04:04:46.358881+0000","last_became_peered":"2024-09-19T04:04:46.358881+0000","last_unstale":"2024-09-19T04:04:49.350389+0000","last_undegraded":"2024-09-19T04:04:49.350389+0000","last_fullsized":"2024-09-19T04:04:49.350389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:58.379187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906237+0000","last_change":"2024-09-19T04:04:39.319705+0000","last_active":"2024-09-19T04:04:49.906237+0000","last_peered":"2024-09-19T04:04:49.906237+0000","last_clean":"2024-09-19T04:04:49.906237+0000","last_became_active":"2024-09-19T04:04:39.318065+0000","last_became_peered":"2024-09-19T04:04:39.318065+0000","last_unstale":"2024-09-19T04:04:49.906237+0000","last_undegraded":"2024-09-19T04:04:49.906237+0000","last_fullsized":"2024-09-19T04:04:49.906237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:15.630927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":66,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:37.629324+0000","last_change":"2024-09-19T04:04:37.215192+0000","last_active":"2024-09-19T04:05:37.629324+0000","last_peered":"2024-09-19T04:05:37.629324+0000","last_clean":"2024-09-19T04:05:37.629324+0000","last_became_active":"2024-09-19T04:04:37.214872+0000","last_became_peered":"2024-09-19T04:04:37.214872+0000","last_unstale":"2024-09-19T04:05:37.629324+0000","last_undegraded":"2024-09-19T04:05:37.629324+0000","last_fullsized":"2024-09-19T04:05:37.629324+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:18.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884852+0000","last_change":"2024-09-19T04:04:46.358113+0000","last_active":"2024-09-19T04:04:49.884852+0000","last_peered":"2024-09-19T04:04:49.884852+0000","last_clean":"2024-09-19T04:04:49.884852+0000","last_became_active":"2024-09-19T04:04:46.356604+0000","last_became_peered":"2024-09-19T04:04:46.356604+0000","last_unstale":"2024-09-19T04:04:49.884852+0000","last_undegraded":"2024-09-19T04:04:49.884852+0000","last_fullsized":"2024-09-19T04:04:49.884852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:38.147738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905726+0000","last_change":"2024-09-19T04:04:46.354516+0000","last_active":"2024-09-19T04:04:49.905726+0000","last_peered":"2024-09-19T04:04:49.905726+0000","last_clean":"2024-09-19T04:04:49.905726+0000","last_became_active":"2024-09-19T04:04:46.354034+0000","last_became_peered":"2024-09-19T04:04:46.354034+0000","last_unstale":"2024-09-19T04:04:49.905726+0000","last_undegraded":"2024-09-19T04:04:49.905726+0000","last_fullsized":"2024-09-19T04:04:49.905726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:21.946746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350792+0000","last_change":"2024-09-19T04:04:39.311324+0000","last_active":"2024-09-19T04:04:49.350792+0000","last_peered":"2024-09-19T04:04:49.350792+0000","last_clean":"2024-09-19T04:04:49.350792+0000","last_became_active":"2024-09-19T04:04:39.310977+0000","last_became_peered":"2024-09-19T04:04:39.310977+0000","last_unstale":"2024-09-19T04:04:49.350792+0000","last_undegraded":"2024-09-19T04:04:49.350792+0000","last_fullsized":"2024-09-19T04:04:49.350792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:43.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:22.530118+0000","last_change":"2024-09-19T04:04:37.212773+0000","last_active":"2024-09-19T04:05:22.530118+0000","last_peered":"2024-09-19T04:05:22.530118+0000","last_clean":"2024-09-19T04:05:22.530118+0000","last_became_active":"2024-09-19T04:04:37.212405+0000","last_became_peered":"2024-09-19T04:04:37.212405+0000","last_unstale":"2024-09-19T04:05:22.530118+0000","last_undegraded":"2024-09-19T04:05:22.530118+0000","last_fullsized":"2024-09-19T04:05:22.530118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:29.020122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905300+0000","last_change":"2024-09-19T04:04:46.363052+0000","last_active":"2024-09-19T04:04:49.905300+0000","last_peered":"2024-09-19T04:04:49.905300+0000","last_clean":"2024-09-19T04:04:49.905300+0000","last_became_active":"2024-09-19T04:04:46.362588+0000","last_became_peered":"2024-09-19T04:04:46.362588+0000","last_unstale":"2024-09-19T04:04:49.905300+0000","last_undegraded":"2024-09-19T04:04:49.905300+0000","last_fullsized":"2024-09-19T04:04:49.905300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:52.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885530+0000","last_change":"2024-09-19T04:04:39.313686+0000","last_active":"2024-09-19T04:04:49.885530+0000","last_peered":"2024-09-19T04:04:49.885530+0000","last_clean":"2024-09-19T04:04:49.885530+0000","last_became_active":"2024-09-19T04:04:39.313386+0000","last_became_peered":"2024-09-19T04:04:39.313386+0000","last_unstale":"2024-09-19T04:04:49.885530+0000","last_undegraded":"2024-09-19T04:04:49.885530+0000","last_fullsized":"2024-09-19T04:04:49.885530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:08.274930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:20.508029+0000","last_change":"2024-09-19T04:04:37.207926+0000","last_active":"2024-09-19T04:05:20.508029+0000","last_peered":"2024-09-19T04:05:20.508029+0000","last_clean":"2024-09-19T04:05:20.508029+0000","last_became_active":"2024-09-19T04:04:37.207538+0000","last_became_peered":"2024-09-19T04:04:37.207538+0000","last_unstale":"2024-09-19T04:05:20.508029+0000","last_undegraded":"2024-09-19T04:05:20.508029+0000","last_fullsized":"2024-09-19T04:05:20.508029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:29.621638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:21.514391+0000","last_change":"2024-09-19T04:04:37.212829+0000","last_active":"2024-09-19T04:05:21.514391+0000","last_peered":"2024-09-19T04:05:21.514391+0000","last_clean":"2024-09-19T04:05:21.514391+0000","last_became_active":"2024-09-19T04:04:37.212294+0000","last_became_peered":"2024-09-19T04:04:37.212294+0000","last_unstale":"2024-09-19T04:05:21.514391+0000","last_undegraded":"2024-09-19T04:05:21.514391+0000","last_fullsized":"2024-09-19T04:05:21.514391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:34.479708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905815+0000","last_change":"2024-09-19T04:04:39.328622+0000","last_active":"2024-09-19T04:04:49.905815+0000","last_peered":"2024-09-19T04:04:49.905815+0000","last_clean":"2024-09-19T04:04:49.905815+0000","last_became_active":"2024-09-19T04:04:39.328290+0000","last_became_peered":"2024-09-19T04:04:39.328290+0000","last_unstale":"2024-09-19T04:04:49.905815+0000","last_undegraded":"2024-09-19T04:04:49.905815+0000","last_fullsized":"2024-09-19T04:04:49.905815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:04.290767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351654+0000","last_change":"2024-09-19T04:04:46.362853+0000","last_active":"2024-09-19T04:04:49.351654+0000","last_peered":"2024-09-19T04:04:49.351654+0000","last_clean":"2024-09-19T04:04:49.351654+0000","last_became_active":"2024-09-19T04:04:46.362392+0000","last_became_peered":"2024-09-19T04:04:46.362392+0000","last_unstale":"2024-09-19T04:04:49.351654+0000","last_undegraded":"2024-09-19T04:04:49.351654+0000","last_fullsized":"2024-09-19T04:04:49.351654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:40.016502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:05:33.625173+0000","last_change":"2024-09-19T04:04:37.213248+0000","last_active":"2024-09-19T04:05:33.625173+0000","last_peered":"2024-09-19T04:05:33.625173+0000","last_clean":"2024-09-19T04:05:33.625173+0000","last_became_active":"2024-09-19T04:04:37.212639+0000","last_became_peered":"2024-09-19T04:04:37.212639+0000","last_unstale":"2024-09-19T04:05:33.625173+0000","last_undegraded":"2024-09-19T04:05:33.625173+0000","last_fullsized":"2024-09-19T04:05:33.625173+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:45:05.917792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904639+0000","last_change":"2024-09-19T04:04:39.330160+0000","last_active":"2024-09-19T04:04:49.904639+0000","last_peered":"2024-09-19T04:04:49.904639+0000","last_clean":"2024-09-19T04:04:49.904639+0000","last_became_active":"2024-09-19T04:04:39.329828+0000","last_became_peered":"2024-09-19T04:04:39.329828+0000","last_unstale":"2024-09-19T04:04:49.904639+0000","last_undegraded":"2024-09-19T04:04:49.904639+0000","last_fullsized":"2024-09-19T04:04:49.904639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:57.712550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.352046+0000","last_change":"2024-09-19T04:04:46.360459+0000","last_active":"2024-09-19T04:04:49.352046+0000","last_peered":"2024-09-19T04:04:49.352046+0000","last_clean":"2024-09-19T04:04:49.352046+0000","last_became_active":"2024-09-19T04:04:46.359914+0000","last_became_peered":"2024-09-19T04:04:46.359914+0000","last_unstale":"2024-09-19T04:04:49.352046+0000","last_undegraded":"2024-09-19T04:04:49.352046+0000","last_fullsized":"2024-09-19T04:04:49.352046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:30.380358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904393+0000","last_change":"2024-09-19T04:04:39.324847+0000","last_active":"2024-09-19T04:04:49.904393+0000","last_peered":"2024-09-19T04:04:49.904393+0000","last_clean":"2024-09-19T04:04:49.904393+0000","last_became_active":"2024-09-19T04:04:39.324402+0000","last_became_peered":"2024-09-19T04:04:39.324402+0000","last_unstale":"2024-09-19T04:04:49.904393+0000","last_undegraded":"2024-09-19T04:04:49.904393+0000","last_fullsized":"2024-09-19T04:04:49.904393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:41.996135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351893+0000","last_change":"2024-09-19T04:04:39.324906+0000","last_active":"2024-09-19T04:04:49.351893+0000","last_peered":"2024-09-19T04:04:49.351893+0000","last_clean":"2024-09-19T04:04:49.351893+0000","last_became_active":"2024-09-19T04:04:39.324552+0000","last_became_peered":"2024-09-19T04:04:39.324552+0000","last_unstale":"2024-09-19T04:04:49.351893+0000","last_undegraded":"2024-09-19T04:04:49.351893+0000","last_fullsized":"2024-09-19T04:04:49.351893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:32:23.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351472+0000","last_change":"2024-09-19T04:04:39.318556+0000","last_active":"2024-09-19T04:04:49.351472+0000","last_peered":"2024-09-19T04:04:49.351472+0000","last_clean":"2024-09-19T04:04:49.351472+0000","last_became_active":"2024-09-19T04:04:39.318200+0000","last_became_peered":"2024-09-19T04:04:39.318200+0000","last_unstale":"2024-09-19T04:04:49.351472+0000","last_undegraded":"2024-09-19T04:04:49.351472+0000","last_fullsized":"2024-09-19T04:04:49.351472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:40.847440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905722+0000","last_change":"2024-09-19T04:04:39.320698+0000","last_active":"2024-09-19T04:04:49.905722+0000","last_peered":"2024-09-19T04:04:49.905722+0000","last_clean":"2024-09-19T04:04:49.905722+0000","last_became_active":"2024-09-19T04:04:39.320419+0000","last_became_peered":"2024-09-19T04:04:39.320419+0000","last_unstale":"2024-09-19T04:04:49.905722+0000","last_undegraded":"2024-09-19T04:04:49.905722+0000","last_fullsized":"2024-09-19T04:04:49.905722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:32.708394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904906+0000","last_change":"2024-09-19T04:04:39.319886+0000","last_active":"2024-09-19T04:04:49.904906+0000","last_peered":"2024-09-19T04:04:49.904906+0000","last_clean":"2024-09-19T04:04:49.904906+0000","last_became_active":"2024-09-19T04:04:39.319217+0000","last_became_peered":"2024-09-19T04:04:39.319217+0000","last_unstale":"2024-09-19T04:04:49.904906+0000","last_undegraded":"2024-09-19T04:04:49.904906+0000","last_fullsized":"2024-09-19T04:04:49.904906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:38.829608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905013+0000","last_change":"2024-09-19T04:04:39.327639+0000","last_active":"2024-09-19T04:04:49.905013+0000","last_peered":"2024-09-19T04:04:49.905013+0000","last_clean":"2024-09-19T04:04:49.905013+0000","last_became_active":"2024-09-19T04:04:39.327349+0000","last_became_peered":"2024-09-19T04:04:39.327349+0000","last_unstale":"2024-09-19T04:04:49.905013+0000","last_undegraded":"2024-09-19T04:04:49.905013+0000","last_fullsized":"2024-09-19T04:04:49.905013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:31.056609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.352008+0000","last_change":"2024-09-19T04:04:39.320253+0000","last_active":"2024-09-19T04:04:49.352008+0000","last_peered":"2024-09-19T04:04:49.352008+0000","last_clean":"2024-09-19T04:04:49.352008+0000","last_became_active":"2024-09-19T04:04:39.319851+0000","last_became_peered":"2024-09-19T04:04:39.319851+0000","last_unstale":"2024-09-19T04:04:49.352008+0000","last_undegraded":"2024-09-19T04:04:49.352008+0000","last_fullsized":"2024-09-19T04:04:49.352008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:53.631216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885400+0000","last_change":"2024-09-19T04:04:39.322779+0000","last_active":"2024-09-19T04:04:49.885400+0000","last_peered":"2024-09-19T04:04:49.885400+0000","last_clean":"2024-09-19T04:04:49.885400+0000","last_became_active":"2024-09-19T04:04:39.322498+0000","last_became_peered":"2024-09-19T04:04:39.322498+0000","last_unstale":"2024-09-19T04:04:49.885400+0000","last_undegraded":"2024-09-19T04:04:49.885400+0000","last_fullsized":"2024-09-19T04:04:49.885400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:24:11.956002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904389+0000","last_change":"2024-09-19T04:04:39.322759+0000","last_active":"2024-09-19T04:04:49.904389+0000","last_peered":"2024-09-19T04:04:49.904389+0000","last_clean":"2024-09-19T04:04:49.904389+0000","last_became_active":"2024-09-19T04:04:39.322429+0000","last_became_peered":"2024-09-19T04:04:39.322429+0000","last_unstale":"2024-09-19T04:04:49.904389+0000","last_undegraded":"2024-09-19T04:04:49.904389+0000","last_fullsized":"2024-09-19T04:04:49.904389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:36.069931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.884890+0000","last_change":"2024-09-19T04:04:39.320078+0000","last_active":"2024-09-19T04:04:49.884890+0000","last_peered":"2024-09-19T04:04:49.884890+0000","last_clean":"2024-09-19T04:04:49.884890+0000","last_became_active":"2024-09-19T04:04:39.319755+0000","last_became_peered":"2024-09-19T04:04:39.319755+0000","last_unstale":"2024-09-19T04:04:49.884890+0000","last_undegraded":"2024-09-19T04:04:49.884890+0000","last_fullsized":"2024-09-19T04:04:49.884890+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:07.651712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906472+0000","last_change":"2024-09-19T04:04:39.319839+0000","last_active":"2024-09-19T04:04:49.906472+0000","last_peered":"2024-09-19T04:04:49.906472+0000","last_clean":"2024-09-19T04:04:49.906472+0000","last_became_active":"2024-09-19T04:04:39.319527+0000","last_became_peered":"2024-09-19T04:04:39.319527+0000","last_unstale":"2024-09-19T04:04:49.906472+0000","last_undegraded":"2024-09-19T04:04:49.906472+0000","last_fullsized":"2024-09-19T04:04:49.906472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:31.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904823+0000","last_change":"2024-09-19T04:04:39.315975+0000","last_active":"2024-09-19T04:04:49.904823+0000","last_peered":"2024-09-19T04:04:49.904823+0000","last_clean":"2024-09-19T04:04:49.904823+0000","last_became_active":"2024-09-19T04:04:39.315671+0000","last_became_peered":"2024-09-19T04:04:39.315671+0000","last_unstale":"2024-09-19T04:04:49.904823+0000","last_undegraded":"2024-09-19T04:04:49.904823+0000","last_fullsized":"2024-09-19T04:04:49.904823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:16.349853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905047+0000","last_change":"2024-09-19T04:04:39.316278+0000","last_active":"2024-09-19T04:04:49.905047+0000","last_peered":"2024-09-19T04:04:49.905047+0000","last_clean":"2024-09-19T04:04:49.905047+0000","last_became_active":"2024-09-19T04:04:39.315876+0000","last_became_peered":"2024-09-19T04:04:39.315876+0000","last_unstale":"2024-09-19T04:04:49.905047+0000","last_undegraded":"2024-09-19T04:04:49.905047+0000","last_fullsized":"2024-09-19T04:04:49.905047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:23.544405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.352016+0000","last_change":"2024-09-19T04:04:39.318952+0000","last_active":"2024-09-19T04:04:49.352016+0000","last_peered":"2024-09-19T04:04:49.352016+0000","last_clean":"2024-09-19T04:04:49.352016+0000","last_became_active":"2024-09-19T04:04:39.318598+0000","last_became_peered":"2024-09-19T04:04:39.318598+0000","last_unstale":"2024-09-19T04:04:49.352016+0000","last_undegraded":"2024-09-19T04:04:49.352016+0000","last_fullsized":"2024-09-19T04:04:49.352016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:12.654611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.350931+0000","last_change":"2024-09-19T04:04:39.311334+0000","last_active":"2024-09-19T04:04:49.350931+0000","last_peered":"2024-09-19T04:04:49.350931+0000","last_clean":"2024-09-19T04:04:49.350931+0000","last_became_active":"2024-09-19T04:04:39.311038+0000","last_became_peered":"2024-09-19T04:04:39.311038+0000","last_unstale":"2024-09-19T04:04:49.350931+0000","last_undegraded":"2024-09-19T04:04:49.350931+0000","last_fullsized":"2024-09-19T04:04:49.350931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:41.655363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905689+0000","last_change":"2024-09-19T04:04:39.319529+0000","last_active":"2024-09-19T04:04:49.905689+0000","last_peered":"2024-09-19T04:04:49.905689+0000","last_clean":"2024-09-19T04:04:49.905689+0000","last_became_active":"2024-09-19T04:04:39.318787+0000","last_became_peered":"2024-09-19T04:04:39.318787+0000","last_unstale":"2024-09-19T04:04:49.905689+0000","last_undegraded":"2024-09-19T04:04:49.905689+0000","last_fullsized":"2024-09-19T04:04:49.905689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.209422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904793+0000","last_change":"2024-09-19T04:04:39.320320+0000","last_active":"2024-09-19T04:04:49.904793+0000","last_peered":"2024-09-19T04:04:49.904793+0000","last_clean":"2024-09-19T04:04:49.904793+0000","last_became_active":"2024-09-19T04:04:39.319091+0000","last_became_peered":"2024-09-19T04:04:39.319091+0000","last_unstale":"2024-09-19T04:04:49.904793+0000","last_undegraded":"2024-09-19T04:04:49.904793+0000","last_fullsized":"2024-09-19T04:04:49.904793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:39.495079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905562+0000","last_change":"2024-09-19T04:04:39.323270+0000","last_active":"2024-09-19T04:04:49.905562+0000","last_peered":"2024-09-19T04:04:49.905562+0000","last_clean":"2024-09-19T04:04:49.905562+0000","last_became_active":"2024-09-19T04:04:39.322890+0000","last_became_peered":"2024-09-19T04:04:39.322890+0000","last_unstale":"2024-09-19T04:04:49.905562+0000","last_undegraded":"2024-09-19T04:04:49.905562+0000","last_fullsized":"2024-09-19T04:04:49.905562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:07.311654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905271+0000","last_change":"2024-09-19T04:04:39.317098+0000","last_active":"2024-09-19T04:04:49.905271+0000","last_peered":"2024-09-19T04:04:49.905271+0000","last_clean":"2024-09-19T04:04:49.905271+0000","last_became_active":"2024-09-19T04:04:39.316792+0000","last_became_peered":"2024-09-19T04:04:39.316792+0000","last_unstale":"2024-09-19T04:04:49.905271+0000","last_undegraded":"2024-09-19T04:04:49.905271+0000","last_fullsized":"2024-09-19T04:04:49.905271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.571149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904697+0000","last_change":"2024-09-19T04:04:39.326473+0000","last_active":"2024-09-19T04:04:49.904697+0000","last_peered":"2024-09-19T04:04:49.904697+0000","last_clean":"2024-09-19T04:04:49.904697+0000","last_became_active":"2024-09-19T04:04:39.325506+0000","last_became_peered":"2024-09-19T04:04:39.325506+0000","last_unstale":"2024-09-19T04:04:49.904697+0000","last_undegraded":"2024-09-19T04:04:49.904697+0000","last_fullsized":"2024-09-19T04:04:49.904697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:40.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905674+0000","last_change":"2024-09-19T04:04:39.323536+0000","last_active":"2024-09-19T04:04:49.905674+0000","last_peered":"2024-09-19T04:04:49.905674+0000","last_clean":"2024-09-19T04:04:49.905674+0000","last_became_active":"2024-09-19T04:04:39.323224+0000","last_became_peered":"2024-09-19T04:04:39.323224+0000","last_unstale":"2024-09-19T04:04:49.905674+0000","last_undegraded":"2024-09-19T04:04:49.905674+0000","last_fullsized":"2024-09-19T04:04:49.905674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:54.640599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.906115+0000","last_change":"2024-09-19T04:04:39.323802+0000","last_active":"2024-09-19T04:04:49.906115+0000","last_peered":"2024-09-19T04:04:49.906115+0000","last_clean":"2024-09-19T04:04:49.906115+0000","last_became_active":"2024-09-19T04:04:39.323031+0000","last_became_peered":"2024-09-19T04:04:39.323031+0000","last_unstale":"2024-09-19T04:04:49.906115+0000","last_undegraded":"2024-09-19T04:04:49.906115+0000","last_fullsized":"2024-09-19T04:04:49.906115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:12.467182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351988+0000","last_change":"2024-09-19T04:04:39.324870+0000","last_active":"2024-09-19T04:04:49.351988+0000","last_peered":"2024-09-19T04:04:49.351988+0000","last_clean":"2024-09-19T04:04:49.351988+0000","last_became_active":"2024-09-19T04:04:39.324551+0000","last_became_peered":"2024-09-19T04:04:39.324551+0000","last_unstale":"2024-09-19T04:04:49.351988+0000","last_undegraded":"2024-09-19T04:04:49.351988+0000","last_fullsized":"2024-09-19T04:04:49.351988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:34.766437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905559+0000","last_change":"2024-09-19T04:04:39.329086+0000","last_active":"2024-09-19T04:04:49.905559+0000","last_peered":"2024-09-19T04:04:49.905559+0000","last_clean":"2024-09-19T04:04:49.905559+0000","last_became_active":"2024-09-19T04:04:39.328651+0000","last_became_peered":"2024-09-19T04:04:39.328651+0000","last_unstale":"2024-09-19T04:04:49.905559+0000","last_undegraded":"2024-09-19T04:04:49.905559+0000","last_fullsized":"2024-09-19T04:04:49.905559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:12.079162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.885021+0000","last_change":"2024-09-19T04:04:39.325427+0000","last_active":"2024-09-19T04:04:49.885021+0000","last_peered":"2024-09-19T04:04:49.885021+0000","last_clean":"2024-09-19T04:04:49.885021+0000","last_became_active":"2024-09-19T04:04:39.325113+0000","last_became_peered":"2024-09-19T04:04:39.325113+0000","last_unstale":"2024-09-19T04:04:49.885021+0000","last_undegraded":"2024-09-19T04:04:49.885021+0000","last_fullsized":"2024-09-19T04:04:49.885021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:15.844503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905172+0000","last_change":"2024-09-19T04:04:39.325032+0000","last_active":"2024-09-19T04:04:49.905172+0000","last_peered":"2024-09-19T04:04:49.905172+0000","last_clean":"2024-09-19T04:04:49.905172+0000","last_became_active":"2024-09-19T04:04:39.324618+0000","last_became_peered":"2024-09-19T04:04:39.324618+0000","last_unstale":"2024-09-19T04:04:49.905172+0000","last_undegraded":"2024-09-19T04:04:49.905172+0000","last_fullsized":"2024-09-19T04:04:49.905172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:58.748613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.351722+0000","last_change":"2024-09-19T04:04:39.318336+0000","last_active":"2024-09-19T04:04:49.351722+0000","last_peered":"2024-09-19T04:04:49.351722+0000","last_clean":"2024-09-19T04:04:49.351722+0000","last_became_active":"2024-09-19T04:04:39.317032+0000","last_became_peered":"2024-09-19T04:04:39.317032+0000","last_unstale":"2024-09-19T04:04:49.351722+0000","last_undegraded":"2024-09-19T04:04:49.351722+0000","last_fullsized":"2024-09-19T04:04:49.351722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:04.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.904393+0000","last_change":"2024-09-19T04:04:39.321909+0000","last_active":"2024-09-19T04:04:49.904393+0000","last_peered":"2024-09-19T04:04:49.904393+0000","last_clean":"2024-09-19T04:04:49.904393+0000","last_became_active":"2024-09-19T04:04:39.320522+0000","last_became_peered":"2024-09-19T04:04:39.320522+0000","last_unstale":"2024-09-19T04:04:49.904393+0000","last_undegraded":"2024-09-19T04:04:49.904393+0000","last_fullsized":"2024-09-19T04:04:49.904393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:57.270998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905574+0000","last_change":"2024-09-19T04:04:39.324610+0000","last_active":"2024-09-19T04:04:49.905574+0000","last_peered":"2024-09-19T04:04:49.905574+0000","last_clean":"2024-09-19T04:04:49.905574+0000","last_became_active":"2024-09-19T04:04:39.324190+0000","last_became_peered":"2024-09-19T04:04:39.324190+0000","last_unstale":"2024-09-19T04:04:49.905574+0000","last_undegraded":"2024-09-19T04:04:49.905574+0000","last_fullsized":"2024-09-19T04:04:49.905574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:51.418210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905278+0000","last_change":"2024-09-19T04:04:39.324220+0000","last_active":"2024-09-19T04:04:49.905278+0000","last_peered":"2024-09-19T04:04:49.905278+0000","last_clean":"2024-09-19T04:04:49.905278+0000","last_became_active":"2024-09-19T04:04:39.323890+0000","last_became_peered":"2024-09-19T04:04:39.323890+0000","last_unstale":"2024-09-19T04:04:49.905278+0000","last_undegraded":"2024-09-19T04:04:49.905278+0000","last_fullsized":"2024-09-19T04:04:49.905278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:15.475088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905779+0000","last_change":"2024-09-19T04:04:39.317240+0000","last_active":"2024-09-19T04:04:49.905779+0000","last_peered":"2024-09-19T04:04:49.905779+0000","last_clean":"2024-09-19T04:04:49.905779+0000","last_became_active":"2024-09-19T04:04:39.316740+0000","last_became_peered":"2024-09-19T04:04:39.316740+0000","last_unstale":"2024-09-19T04:04:49.905779+0000","last_undegraded":"2024-09-19T04:04:49.905779+0000","last_fullsized":"2024-09-19T04:04:49.905779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:46.705467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T04:04:49.905586+0000","last_change":"2024-09-19T04:04:39.319665+0000","last_active":"2024-09-19T04:04:49.905586+0000","last_peered":"2024-09-19T04:04:49.905586+0000","last_clean":"2024-09-19T04:04:49.905586+0000","last_became_active":"2024-09-19T04:04:39.319227+0000","last_became_peered":"2024-09-19T04:04:39.319227+0000","last_unstale":"2024-09-19T04:04:49.905586+0000","last_undegraded":"2024-09-19T04:04:49.905586+0000","last_fullsized":"2024-09-19T04:04:49.905586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:45.581254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63488,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":186,"num_read_kb":168,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":177136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248101,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":491318,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710749,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":1134807,"data_compressed":53410,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874275,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":495344,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":491626,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":1088125,"data_compressed":53410,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":487145,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1083314,"data_compressed":53410,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":490826,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":486460,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":544675,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":531520,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":488208,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:05:39.719 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:05:39.719 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T04:05:39.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T04:05:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:40.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:40 smithi019 ceph-mon[30938]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-09-19T04:05:40.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:40 smithi019 ceph-mon[30938]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:05:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 -- 172.21.15.19:0/2334879684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 msgr2=0x7f1a3c104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 --2- 172.21.15.19:0/2334879684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 0x7f1a3c104b10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009900 tx=0x7f1a3802a410 comp rx=0 tx=0).stop 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 -- 172.21.15.19:0/2334879684 shutdown_connections 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 --2- 172.21.15.19:0/2334879684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3c107980 0x7f1a3c109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 --2- 172.21.15.19:0/2334879684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a3c105050 0x7f1a3c107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 --2- 172.21.15.19:0/2334879684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 0x7f1a3c104b10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 -- 172.21.15.19:0/2334879684 >> 172.21.15.19:0/2334879684 conn(0x7f1a3c0fc2a0 msgr2=0x7f1a3c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 -- 172.21.15.19:0/2334879684 shutdown_connections 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 -- 172.21.15.19:0/2334879684 wait complete. 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 Processor -- start 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.455+0000 7f1a4391b640 1 -- start start 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a4391b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 0x7f1a3c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a4391b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3c105050 0x7f1a3c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a4391b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a3c107980 0x7f1a3c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a4391b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c10d900 con 0x7f1a3c102720 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a4391b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c10d780 con 0x7f1a3c107980 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a4391b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c10da80 con 0x7f1a3c105050 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a416b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 0x7f1a3c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a416b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 0x7f1a3c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56332/0 (socket says 172.21.15.19:56332) 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a416b7640 1 -- 172.21.15.19:0/3395982404 learned_addr learned my addr 172.21.15.19:0/3395982404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a40eb6640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3c105050 0x7f1a3c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a41eb8640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a3c107980 0x7f1a3c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a416b7640 1 -- 172.21.15.19:0/3395982404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3c105050 msgr2=0x7f1a3c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a416b7640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3c105050 0x7f1a3c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a416b7640 1 -- 172.21.15.19:0/3395982404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a3c107980 msgr2=0x7f1a3c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a416b7640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a3c107980 0x7f1a3c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.456+0000 7f1a416b7640 1 -- 172.21.15.19:0/3395982404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c19fbb0 con 0x7f1a3c102720 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.457+0000 7f1a416b7640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 0x7f1a3c197f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009230 tx=0x7f1a38004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.457+0000 7f1a327fc640 1 -- 172.21.15.19:0/3395982404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3803f030 con 0x7f1a3c102720 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.457+0000 7f1a327fc640 1 -- 172.21.15.19:0/3395982404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a38003ce0 con 0x7f1a3c102720 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.457+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c19fe40 con 0x7f1a3c102720 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.457+0000 7f1a327fc640 1 -- 172.21.15.19:0/3395982404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38035070 con 0x7f1a3c102720 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.457+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c1a0310 con 0x7f1a3c102720 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.458+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3c0fdd90 con 0x7f1a3c102720 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.459+0000 7f1a327fc640 1 -- 172.21.15.19:0/3395982404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1a38042030 con 0x7f1a3c102720 2024-09-19T04:05:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.460+0000 7f1a327fc640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1a280a3380 0x7f1a280a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:40.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.460+0000 7f1a40eb6640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1a280a3380 0x7f1a280a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:40.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.460+0000 7f1a327fc640 1 -- 172.21.15.19:0/3395982404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1a380f0290 con 0x7f1a3c102720 2024-09-19T04:05:40.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.460+0000 7f1a40eb6640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1a280a3380 0x7f1a280a5840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c1994a0 tx=0x7f1a2c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:40.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.464+0000 7f1a327fc640 1 -- 172.21.15.19:0/3395982404 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1a380bc5c0 con 0x7f1a3c102720 2024-09-19T04:05:40.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:40 smithi111 ceph-mon[35365]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-09-19T04:05:40.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:40 smithi111 ceph-mon[35365]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:05:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:40 smithi196 ceph-mon[35731]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-09-19T04:05:40.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:40 smithi196 ceph-mon[35731]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:05:40.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.711+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1a3c02da10 con 0x7f1a3c102720 2024-09-19T04:05:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.714+0000 7f1a327fc640 1 -- 172.21.15.19:0/3395982404 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1a380c08c0 con 0x7f1a3c102720 2024-09-19T04:05:40.715 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-09-19T04:05:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQAUo+tmRQt9KhAApDWjQsBA9BZ0B4j6W/HqIg== 2024-09-19T04:05:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1a280a3380 msgr2=0x7f1a280a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1a280a3380 0x7f1a280a5840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c1994a0 tx=0x7f1a2c00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 msgr2=0x7f1a3c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 0x7f1a3c197f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009230 tx=0x7f1a38004430 comp rx=0 tx=0).stop 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 shutdown_connections 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1a280a3380 0x7f1a280a5840 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a3c107980 0x7f1a3c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3c105050 0x7f1a3c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 --2- 172.21.15.19:0/3395982404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3c102720 0x7f1a3c197f80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 >> 172.21.15.19:0/3395982404 conn(0x7f1a3c0fc2a0 msgr2=0x7f1a3c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 shutdown_connections 2024-09-19T04:05:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:40.716+0000 7f1a4391b640 1 -- 172.21.15.19:0/3395982404 wait complete. 2024-09-19T04:05:41.407 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T04:05:41.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T04:05:41.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T04:05:41.446 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T04:05:41.446 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T04:05:41.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T04:05:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2024-09-19T04:05:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T04:05:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-19T04:05:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T04:05:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-19T04:05:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 1]}]: dispatch 2024-09-19T04:05:41.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-19T04:05:41.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-09-19T04:05:41.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-19T04:05:41.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3395982404' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:05:41.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:41 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3395982404' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:05:41.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2024-09-19T04:05:41.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T04:05:41.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-19T04:05:41.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T04:05:41.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-19T04:05:41.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 1]}]: dispatch 2024-09-19T04:05:41.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-19T04:05:41.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-09-19T04:05:41.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-19T04:05:41.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3395982404' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:05:41.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:41 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3395982404' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:05:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2024-09-19T04:05:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T04:05:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-19T04:05:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T04:05:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-19T04:05:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 1]}]: dispatch 2024-09-19T04:05:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-19T04:05:41.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-09-19T04:05:41.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-19T04:05:41.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3395982404' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T04:05:41.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:41 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3395982404' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T04:05:41.743 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2409744459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8100e20 msgr2=0x7fb4d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2409744459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8100e20 0x7fb4d8101200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009900 tx=0x7fb4c802a480 comp rx=0 tx=0).stop 2024-09-19T04:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2409744459 shutdown_connections 2024-09-19T04:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2409744459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d8102ef0 0x7fb4d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2409744459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d8102570 0x7fb4d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2409744459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8100e20 0x7fb4d8101200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2409744459 >> 172.21.15.19:0/2409744459 conn(0x7fb4d80fc7d0 msgr2=0x7fb4d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2409744459 shutdown_connections 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2409744459 wait complete. 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.197+0000 7fb4ddf47640 1 Processor -- start 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4ddf47640 1 -- start start 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4ddf47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d8100e20 0x7fb4d81993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4ddf47640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d8102570 0x7fb4d8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4ddf47640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8102ef0 0x7fb4d81a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4ddf47640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8110f50 con 0x7fb4d8102ef0 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4ddf47640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8110dd0 con 0x7fb4d8102570 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4ddf47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81110d0 con 0x7fb4d8100e20 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8102ef0 0x7fb4d81a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8102ef0 0x7fb4d81a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37852/0 (socket says 172.21.15.19:37852) 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d7fff640 1 -- 172.21.15.19:0/2230661052 learned_addr learned my addr 172.21.15.19:0/2230661052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4cffff640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d8102570 0x7fb4d8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d8100e20 0x7fb4d81993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d7fff640 1 -- 172.21.15.19:0/2230661052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d8100e20 msgr2=0x7fb4d81993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d8100e20 0x7fb4d81993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d7fff640 1 -- 172.21.15.19:0/2230661052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d8102570 msgr2=0x7fb4d8199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d8102570 0x7fb4d8199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.198+0000 7fb4d7fff640 1 -- 172.21.15.19:0/2230661052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d81a4980 con 0x7fb4d8102ef0 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.199+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8102ef0 0x7fb4d81a4440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000eaa0 tx=0x7fb4d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.199+0000 7fb4d57fa640 1 -- 172.21.15.19:0/2230661052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000cdb0 con 0x7fb4d8102ef0 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.199+0000 7fb4d57fa640 1 -- 172.21.15.19:0/2230661052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d0005020 con 0x7fb4d8102ef0 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.199+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d81a4b50 con 0x7fb4d8102ef0 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.199+0000 7fb4d57fa640 1 -- 172.21.15.19:0/2230661052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00116b0 con 0x7fb4d8102ef0 2024-09-19T04:05:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.199+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81a5060 con 0x7fb4d8102ef0 2024-09-19T04:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.200+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d8101200 con 0x7fb4d8102ef0 2024-09-19T04:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.201+0000 7fb4d57fa640 1 -- 172.21.15.19:0/2230661052 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb4d0005560 con 0x7fb4d8102ef0 2024-09-19T04:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.202+0000 7fb4d57fa640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb4c40a3350 0x7fb4c40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.202+0000 7fb4d57fa640 1 -- 172.21.15.19:0/2230661052 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fb4d00d0290 con 0x7fb4d8102ef0 2024-09-19T04:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.202+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb4c40a3350 0x7fb4c40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.202+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb4c40a3350 0x7fb4c40a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80096c0 tx=0x7fb4c8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.205+0000 7fb4d57fa640 1 -- 172.21.15.19:0/2230661052 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4d0016030 con 0x7fb4d8102ef0 2024-09-19T04:05:42.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T04:05:42.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2024-09-19T04:05:42.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T04:05:42.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-19T04:05:42.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T04:05:42.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-19T04:05:42.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 1]}]': finished 2024-09-19T04:05:42.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-19T04:05:42.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-19T04:05:42.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-19T04:05:42.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:42 smithi019 ceph-mon[30938]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:05:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.432+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb4d806a560 con 0x7fb4d8102ef0 2024-09-19T04:05:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.437+0000 7fb4d57fa640 1 -- 172.21.15.19:0/2230661052 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264050 (secure 0 0 0) 0x7fb4d806a560 con 0x7fb4d8102ef0 2024-09-19T04:05:42.441 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:42.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb4c40a3350 msgr2=0x7fb4c40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:42.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb4c40a3350 0x7fb4c40a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80096c0 tx=0x7fb4c8000bc0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8102ef0 msgr2=0x7fb4d81a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:42.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8102ef0 0x7fb4d81a4440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000eaa0 tx=0x7fb4d000ef60 comp rx=0 tx=0).stop 2024-09-19T04:05:42.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 shutdown_connections 2024-09-19T04:05:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb4c40a3350 0x7fb4c40a5810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8102ef0 0x7fb4d81a4440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d8102570 0x7fb4d8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 --2- 172.21.15.19:0/2230661052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d8100e20 0x7fb4d81993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 >> 172.21.15.19:0/2230661052 conn(0x7fb4d80fc7d0 msgr2=0x7fb4d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 shutdown_connections 2024-09-19T04:05:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:42.440+0000 7fb4ddf47640 1 -- 172.21.15.19:0/2230661052 wait complete. 2024-09-19T04:05:42.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 1]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-19T04:05:42.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:42 smithi111 ceph-mon[35365]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:05:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 1]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-19T04:05:42.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:42 smithi196 ceph-mon[35731]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T04:05:43.152 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":2387193681},{"type":"v1","addr":"172.21.15.19:6801","nonce":2387193681}]},"active_addr":"172.21.15.19:6801/2387193681","active_change":"2024-09-19T03:57:40.326691+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2224928695}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2156450853}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1548003543}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1839666147}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1437886969}]}]} 2024-09-19T04:05:43.157 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T04:05:43.157 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T04:05:43.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:05:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:43.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:43 smithi019 ceph-mon[30938]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:05:43.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:43 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2230661052' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:05:43.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:43 smithi111 ceph-mon[35365]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:05:43.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:43 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2230661052' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:05:43.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:43 smithi196 ceph-mon[35731]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T04:05:43.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:43 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2230661052' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T04:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.897+0000 7fd5ee22c640 1 -- 172.21.15.19:0/479681867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8102d90 msgr2=0x7fd5e8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.897+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/479681867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8102d90 0x7fd5e8103170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009930 tx=0x7fd5d802a420 comp rx=0 tx=0).stop 2024-09-19T04:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 -- 172.21.15.19:0/479681867 shutdown_connections 2024-09-19T04:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/479681867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5e8108290 0x7fd5e810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/479681867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e8103740 0x7fd5e8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/479681867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8102d90 0x7fd5e8103170 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 -- 172.21.15.19:0/479681867 >> 172.21.15.19:0/479681867 conn(0x7fd5e80fc7d0 msgr2=0x7fd5e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 -- 172.21.15.19:0/479681867 shutdown_connections 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 -- 172.21.15.19:0/479681867 wait complete. 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 Processor -- start 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.898+0000 7fd5ee22c640 1 -- start start 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5ee22c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5e8102d90 0x7fd5e81009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5ee22c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e8103740 0x7fd5e8100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5ee22c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8108290 0x7fd5e8101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5ee22c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8112090 con 0x7fd5e8108290 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5ee22c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8111f10 con 0x7fd5e8103740 2024-09-19T04:05:43.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5ee22c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8112210 con 0x7fd5e8102d90 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8108290 0x7fd5e8101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8108290 0x7fd5e8101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37860/0 (socket says 172.21.15.19:37860) 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 -- 172.21.15.19:0/1858007537 learned_addr learned my addr 172.21.15.19:0/1858007537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e77fe640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5e8102d90 0x7fd5e81009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e6ffd640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e8103740 0x7fd5e8100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 -- 172.21.15.19:0/1858007537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5e8102d90 msgr2=0x7fd5e81009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5e8102d90 0x7fd5e81009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 -- 172.21.15.19:0/1858007537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e8103740 msgr2=0x7fd5e8100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e8103740 0x7fd5e8100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 -- 172.21.15.19:0/1858007537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e81a69d0 con 0x7fd5e8108290 2024-09-19T04:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e6ffd640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e8103740 0x7fd5e8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e7fff640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8108290 0x7fd5e8101470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000eaa0 tx=0x7fd5e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e4ff9640 1 -- 172.21.15.19:0/1858007537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000cdb0 con 0x7fd5e8108290 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e4ff9640 1 -- 172.21.15.19:0/1858007537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e0005020 con 0x7fd5e8108290 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e81a6cc0 con 0x7fd5e8108290 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.899+0000 7fd5e4ff9640 1 -- 172.21.15.19:0/1858007537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00116b0 con 0x7fd5e8108290 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.900+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81a7200 con 0x7fd5e8108290 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.900+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e806a7f0 con 0x7fd5e8108290 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.902+0000 7fd5e4ff9640 1 -- 172.21.15.19:0/1858007537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd5e0005560 con 0x7fd5e8108290 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.902+0000 7fd5e4ff9640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5d00a32d0 0x7fd5d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.902+0000 7fd5e4ff9640 1 -- 172.21.15.19:0/1858007537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fd5e00cffc0 con 0x7fd5e8108290 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.902+0000 7fd5e77fe640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5d00a32d0 0x7fd5d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.903+0000 7fd5e77fe640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5d00a32d0 0x7fd5d00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009900 tx=0x7fd5d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:43.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:43.907+0000 7fd5e4ff9640 1 -- 172.21.15.19:0/1858007537 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd5e0016030 con 0x7fd5e8108290 2024-09-19T04:05:44.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.083+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5e810a2a0 con 0x7fd5e8108290 2024-09-19T04:05:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.086+0000 7fd5e4ff9640 1 -- 172.21.15.19:0/1858007537 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24685 (secure 0 0 0) 0x7fd5e009c840 con 0x7fd5e8108290 2024-09-19T04:05:44.087 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:44.088 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:05:42.327059+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5d00a32d0 msgr2=0x7fd5d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5d00a32d0 0x7fd5d00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009900 tx=0x7fd5d80058d0 comp rx=0 tx=0).stop 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8108290 msgr2=0x7fd5e8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8108290 0x7fd5e8101470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000eaa0 tx=0x7fd5e000ef60 comp rx=0 tx=0).stop 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 shutdown_connections 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5d00a32d0 0x7fd5d00a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5e8108290 0x7fd5e8101470 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e8103740 0x7fd5e8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 --2- 172.21.15.19:0/1858007537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5e8102d90 0x7fd5e81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 >> 172.21.15.19:0/1858007537 conn(0x7fd5e80fc7d0 msgr2=0x7fd5e8106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 shutdown_connections 2024-09-19T04:05:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:44.088+0000 7fd5ee22c640 1 -- 172.21.15.19:0/1858007537 wait complete. 2024-09-19T04:05:44.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:44 smithi111 ceph-mon[35365]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T04:05:44.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:44 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1858007537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:44.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:44 smithi019 ceph-mon[30938]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T04:05:44.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:44 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1858007537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:44.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:44 smithi196 ceph-mon[35731]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T04:05:44.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:44 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1858007537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:44.812 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T04:05:44.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:45.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 -- 172.21.15.19:0/3277311984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868102790 msgr2=0x7f98681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 --2- 172.21.15.19:0/3277311984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868102790 0x7f98681067e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f985c009900 tx=0x7f985c02a410 comp rx=0 tx=0).stop 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 -- 172.21.15.19:0/3277311984 shutdown_connections 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 --2- 172.21.15.19:0/3277311984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9868106f10 0x7f986810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 --2- 172.21.15.19:0/3277311984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868102790 0x7f98681067e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 --2- 172.21.15.19:0/3277311984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9868101de0 0x7f98681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 -- 172.21.15.19:0/3277311984 >> 172.21.15.19:0/3277311984 conn(0x7f98680fd8c0 msgr2=0x7f98680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 -- 172.21.15.19:0/3277311984 shutdown_connections 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.573+0000 7f986d198640 1 -- 172.21.15.19:0/3277311984 wait complete. 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.574+0000 7f986d198640 1 Processor -- start 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986d198640 1 -- start start 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986d198640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868101de0 0x7f98681984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986d198640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9868102790 0x7f9868198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986d198640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9868106f10 0x7f986819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:45.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986d198640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9868111f70 con 0x7f9868101de0 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986d198640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9868111df0 con 0x7f9868106f10 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986d198640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98681120f0 con 0x7f9868102790 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f9866d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868101de0 0x7f98681984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f9866d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868101de0 0x7f98681984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37874/0 (socket says 172.21.15.19:37874) 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f9866d9d640 1 -- 172.21.15.19:0/3251888159 learned_addr learned my addr 172.21.15.19:0/3251888159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986659c640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9868102790 0x7f9868198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f986759e640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9868106f10 0x7f986819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f9866d9d640 1 -- 172.21.15.19:0/3251888159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9868102790 msgr2=0x7f9868198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f9866d9d640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9868102790 0x7f9868198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f9866d9d640 1 -- 172.21.15.19:0/3251888159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9868106f10 msgr2=0x7f986819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f9866d9d640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9868106f10 0x7f986819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.575+0000 7f9866d9d640 1 -- 172.21.15.19:0/3251888159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986819d480 con 0x7f9868101de0 2024-09-19T04:05:45.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.576+0000 7f9866d9d640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868101de0 0x7f98681984d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f985800e9f0 tx=0x7f985800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.576+0000 7f984ffff640 1 -- 172.21.15.19:0/3251888159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985800cd80 con 0x7f9868101de0 2024-09-19T04:05:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.576+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986819d770 con 0x7f9868101de0 2024-09-19T04:05:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.576+0000 7f984ffff640 1 -- 172.21.15.19:0/3251888159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9858010910 con 0x7f9868101de0 2024-09-19T04:05:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.576+0000 7f984ffff640 1 -- 172.21.15.19:0/3251888159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580188f0 con 0x7f9868101de0 2024-09-19T04:05:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.576+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a5050 con 0x7f9868101de0 2024-09-19T04:05:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.577+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98681021c0 con 0x7f9868101de0 2024-09-19T04:05:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.578+0000 7f984ffff640 1 -- 172.21.15.19:0/3251888159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9858010ac0 con 0x7f9868101de0 2024-09-19T04:05:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.579+0000 7f984ffff640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98500a32d0 0x7f98500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.579+0000 7f986659c640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98500a32d0 0x7f98500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.579+0000 7f984ffff640 1 -- 172.21.15.19:0/3251888159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f98580d0290 con 0x7f9868101de0 2024-09-19T04:05:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.580+0000 7f986659c640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98500a32d0 0x7f98500a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f985c009230 tx=0x7f985c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.583+0000 7f984ffff640 1 -- 172.21.15.19:0/3251888159 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f985809cb10 con 0x7f9868101de0 2024-09-19T04:05:45.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.762+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f986810a150 con 0x7f9868101de0 2024-09-19T04:05:45.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.764+0000 7f984ffff640 1 -- 172.21.15.19:0/3251888159 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24685 (secure 0 0 0) 0x7f985809c410 con 0x7f9868101de0 2024-09-19T04:05:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:05:45.766 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:05:42.327059+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:05:45.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98500a32d0 msgr2=0x7f98500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98500a32d0 0x7f98500a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f985c009230 tx=0x7f985c0058d0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868101de0 msgr2=0x7f98681984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868101de0 0x7f98681984d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f985800e9f0 tx=0x7f985800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 shutdown_connections 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98500a32d0 0x7f98500a5790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9868106f10 0x7f986819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9868102790 0x7f9868198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 --2- 172.21.15.19:0/3251888159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9868101de0 0x7f98681984d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 >> 172.21.15.19:0/3251888159 conn(0x7f98680fd8c0 msgr2=0x7f98680fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 shutdown_connections 2024-09-19T04:05:45.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:45.766+0000 7f986d198640 1 -- 172.21.15.19:0/3251888159 wait complete. 2024-09-19T04:05:46.414 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T04:05:46.414 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T04:05:46.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T04:05:46.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T04:05:46.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T04:05:46.416 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T04:05:46.416 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T04:05:46.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T04:05:46.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T04:05:46.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T04:05:46.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T04:05:46.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T04:05:46.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:46 smithi111 ceph-mon[35365]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.6 KiB/s wr, 11 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-09-19T04:05:46.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:46 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3251888159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:46.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:46 smithi019 ceph-mon[30938]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.6 KiB/s wr, 11 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-09-19T04:05:46.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:46 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3251888159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:46.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:46 smithi196 ceph-mon[35731]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.6 KiB/s wr, 11 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-09-19T04:05:46.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:46 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3251888159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:05:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.213 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.220 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.220 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.220 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.230 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.230 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:05:47.603 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:47 smithi019 ceph-mon[30938]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 639 B/s wr, 8 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T04:05:47.604 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:47 smithi019 ceph-mon[30938]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-19T04:05:47.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:47 smithi111 ceph-mon[35365]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 639 B/s wr, 8 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T04:05:47.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:47 smithi111 ceph-mon[35365]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-19T04:05:47.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:47 smithi196 ceph-mon[35731]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 639 B/s wr, 8 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T04:05:47.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:47 smithi196 ceph-mon[35731]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-19T04:05:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 -- 172.21.15.19:0/1827552036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4410c170 msgr2=0x7f0e4410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 --2- 172.21.15.19:0/1827552036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4410c170 0x7f0e4410e610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009ca0 tx=0x7f0e3c02a950 comp rx=0 tx=0).stop 2024-09-19T04:05:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 -- 172.21.15.19:0/1827552036 shutdown_connections 2024-09-19T04:05:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 --2- 172.21.15.19:0/1827552036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4410c170 0x7f0e4410e610 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 --2- 172.21.15.19:0/1827552036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e44103660 0x7f0e4410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 --2- 172.21.15.19:0/1827552036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e44102d40 0x7f0e44103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 -- 172.21.15.19:0/1827552036 >> 172.21.15.19:0/1827552036 conn(0x7f0e440fc7a0 msgr2=0x7f0e440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 -- 172.21.15.19:0/1827552036 shutdown_connections 2024-09-19T04:05:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.396+0000 7f0e48edb640 1 -- 172.21.15.19:0/1827552036 wait complete. 2024-09-19T04:05:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e48edb640 1 Processor -- start 2024-09-19T04:05:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e48edb640 1 -- start start 2024-09-19T04:05:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e48edb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e44102d40 0x7f0e44198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e48edb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e44103660 0x7f0e44198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e48edb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4410c170 0x7f0e4419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e48edb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44111f70 con 0x7f0e44102d40 2024-09-19T04:05:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e48edb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44111df0 con 0x7f0e44103660 2024-09-19T04:05:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e48edb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441120f0 con 0x7f0e4410c170 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e44102d40 0x7f0e44198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e44102d40 0x7f0e44198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37880/0 (socket says 172.21.15.19:37880) 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e4259c640 1 -- 172.21.15.19:0/1408511195 learned_addr learned my addr 172.21.15.19:0/1408511195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e41d9b640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e44103660 0x7f0e44198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.397+0000 7f0e42d9d640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4410c170 0x7f0e4419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e4259c640 1 -- 172.21.15.19:0/1408511195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4410c170 msgr2=0x7f0e4419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e4259c640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4410c170 0x7f0e4419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e4259c640 1 -- 172.21.15.19:0/1408511195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e44103660 msgr2=0x7f0e44198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e4259c640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e44103660 0x7f0e44198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e4259c640 1 -- 172.21.15.19:0/1408511195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4419d3c0 con 0x7f0e44102d40 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e4259c640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e44102d40 0x7f0e44198410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c970 tx=0x7f0e3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34007bb0 con 0x7f0e44102d40 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e34007d50 con 0x7f0e44102d40 2024-09-19T04:05:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4419d6b0 con 0x7f0e44102d40 2024-09-19T04:05:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400f450 con 0x7f0e44102d40 2024-09-19T04:05:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.398+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e441a4f90 con 0x7f0e44102d40 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.399+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e440fe750 con 0x7f0e44102d40 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.400+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0e34004fa0 con 0x7f0e44102d40 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.401+0000 7f0e337fe640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e2c0a32d0 0x7f0e2c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.401+0000 7f0e41d9b640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e2c0a32d0 0x7f0e2c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.401+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f0e340cff80 con 0x7f0e44102d40 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.401+0000 7f0e337fe640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] conn(0x7f0e2c0ae420 0x7f0e2c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.401+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 --> [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e2c0b0f90 con 0x7f0e2c0ae420 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.401+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f0e340d03d0 con 0x7f0e44102d40 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.402+0000 7f0e41d9b640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e2c0a32d0 0x7f0e2c0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0e440fecc0 tx=0x7f0e3800b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.402+0000 7f0e42d9d640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] conn(0x7f0e2c0ae420 0x7f0e2c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:48.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.402+0000 7f0e42d9d640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] conn(0x7f0e2c0ae420 0x7f0e2c0b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:48.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.403+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 <== osd.7 v2:172.21.15.111:6816/837241728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e2c0b0f90 con 0x7f0e2c0ae420 2024-09-19T04:05:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.417+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 --> [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e4406b070 con 0x7f0e2c0ae420 2024-09-19T04:05:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e337fe640 1 -- 172.21.15.19:0/1408511195 <== osd.7 v2:172.21.15.111:6816/837241728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0e4406b070 con 0x7f0e2c0ae420 2024-09-19T04:05:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 >> [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] conn(0x7f0e2c0ae420 msgr2=0x7f0e2c0b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] conn(0x7f0e2c0ae420 0x7f0e2c0b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e2c0a32d0 msgr2=0x7f0e2c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e2c0a32d0 0x7f0e2c0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0e440fecc0 tx=0x7f0e3800b2e0 comp rx=0 tx=0).stop 2024-09-19T04:05:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e44102d40 msgr2=0x7f0e44198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 --2- 172.21.15.19:0/1408511195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e44102d40 0x7f0e44198410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c970 tx=0x7f0e3400ce30 comp rx=0 tx=0).stop 2024-09-19T04:05:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e42d9d640 1 -- 172.21.15.19:0/1408511195 reap_dead start 2024-09-19T04:05:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 shutdown_connections 2024-09-19T04:05:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 >> 172.21.15.19:0/1408511195 conn(0x7f0e440fc7a0 msgr2=0x7f0e440fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 shutdown_connections 2024-09-19T04:05:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:48.418+0000 7f0e48edb640 1 -- 172.21.15.19:0/1408511195 wait complete. 2024-09-19T04:05:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:48 smithi111 ceph-mon[35365]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-19T04:05:48.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:48 smithi111 ceph-mon[35365]: Cluster is now healthy 2024-09-19T04:05:48.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:48 smithi019 ceph-mon[30938]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-19T04:05:48.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:48 smithi019 ceph-mon[30938]: Cluster is now healthy 2024-09-19T04:05:48.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:48 smithi196 ceph-mon[35731]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-19T04:05:48.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:48 smithi196 ceph-mon[35731]: Cluster is now healthy 2024-09-19T04:05:49.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:49 smithi019 ceph-mon[30938]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 11 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T04:05:49.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4058336852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102570 msgr2=0x7f86a41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 --2- 172.21.15.19:0/4058336852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102570 0x7f86a41029b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8698009ca0 tx=0x7f869802a950 comp rx=0 tx=0).stop 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4058336852 shutdown_connections 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 --2- 172.21.15.19:0/4058336852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a4102ef0 0x7f86a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 --2- 172.21.15.19:0/4058336852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102570 0x7f86a41029b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 --2- 172.21.15.19:0/4058336852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86a4100e20 0x7f86a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4058336852 >> 172.21.15.19:0/4058336852 conn(0x7f86a40fc7d0 msgr2=0x7f86a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4058336852 shutdown_connections 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.606+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4058336852 wait complete. 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a9f3e640 1 Processor -- start 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a9f3e640 1 -- start start 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a9f3e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86a4100e20 0x7f86a4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:49.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a9f3e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a4102570 0x7f86a4199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a9f3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102ef0 0x7f86a41a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a9f3e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4110f80 con 0x7f86a4102ef0 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a9f3e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4110e00 con 0x7f86a4100e20 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a9f3e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4111100 con 0x7f86a4102570 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102ef0 0x7f86a41a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102ef0 0x7f86a41a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37930/0 (socket says 172.21.15.19:37930) 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a3fff640 1 -- 172.21.15.19:0/4288461524 learned_addr learned my addr 172.21.15.19:0/4288461524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a37fe640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86a4100e20 0x7f86a4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.607+0000 7f86a2ffd640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a4102570 0x7f86a4199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a3fff640 1 -- 172.21.15.19:0/4288461524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a4102570 msgr2=0x7f86a4199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a3fff640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a4102570 0x7f86a4199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a3fff640 1 -- 172.21.15.19:0/4288461524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86a4100e20 msgr2=0x7f86a4199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a3fff640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86a4100e20 0x7f86a4199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a3fff640 1 -- 172.21.15.19:0/4288461524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a41a4910 con 0x7f86a4102ef0 2024-09-19T04:05:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a3fff640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102ef0 0x7f86a41a43d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f869c00c970 tx=0x7f869c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c007bb0 con 0x7f86a4102ef0 2024-09-19T04:05:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c007d50 con 0x7f86a4102ef0 2024-09-19T04:05:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a41a4ae0 con 0x7f86a4102ef0 2024-09-19T04:05:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c00f450 con 0x7f86a4102ef0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.608+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a41a4ff0 con 0x7f86a4102ef0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.609+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86a4062980 con 0x7f86a4102ef0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.610+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f869c004ee0 con 0x7f86a4102ef0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.611+0000 7f86a0ff9640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f868c0a32d0 0x7f868c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.611+0000 7f86a37fe640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f868c0a32d0 0x7f868c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.611+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f869c0cfd20 con 0x7f86a4102ef0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.611+0000 7f86a0ff9640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] conn(0x7f868c0ae420 0x7f868c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.611+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 --> [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f869c012ec0 con 0x7f868c0ae420 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.611+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f869c0d0170 con 0x7f86a4102ef0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.612+0000 7f86a37fe640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f868c0a32d0 0x7f868c0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8694005d00 tx=0x7f869400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.612+0000 7f86a2ffd640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] conn(0x7f868c0ae420 0x7f868c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.612+0000 7f86a2ffd640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] conn(0x7f868c0ae420 0x7f868c0b08c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.613+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 <== osd.4 v2:172.21.15.111:6808/942057165 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f869c012ec0 con 0x7f868c0ae420 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.626+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 --> [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86a4062e00 con 0x7f868c0ae420 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.627+0000 7f86a0ff9640 1 -- 172.21.15.19:0/4288461524 <== osd.4 v2:172.21.15.111:6808/942057165 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f86a4062e00 con 0x7f868c0ae420 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.627+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 >> [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] conn(0x7f868c0ae420 msgr2=0x7f868c0b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.627+0000 7f86a9f3e640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] conn(0x7f868c0ae420 0x7f868c0b08c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f868c0a32d0 msgr2=0x7f868c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a9f3e640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f868c0a32d0 0x7f868c0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8694005d00 tx=0x7f869400b410 comp rx=0 tx=0).stop 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102ef0 msgr2=0x7f86a41a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a9f3e640 1 --2- 172.21.15.19:0/4288461524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86a4102ef0 0x7f86a41a43d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f869c00c970 tx=0x7f869c00ce30 comp rx=0 tx=0).stop 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a3fff640 1 -- 172.21.15.19:0/4288461524 reap_dead start 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 shutdown_connections 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 >> 172.21.15.19:0/4288461524 conn(0x7f86a40fc7d0 msgr2=0x7f86a4106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 shutdown_connections 2024-09-19T04:05:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.628+0000 7f86a9f3e640 1 -- 172.21.15.19:0/4288461524 wait complete. 2024-09-19T04:05:49.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:49 smithi111 ceph-mon[35365]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 11 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T04:05:49.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:49 smithi196 ceph-mon[35731]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 11 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T04:05:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3110621601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4102790 msgr2=0x7f4bd41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 --2- 172.21.15.19:0/3110621601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4102790 0x7f4bd41067e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009900 tx=0x7f4bc402a410 comp rx=0 tx=0).stop 2024-09-19T04:05:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3110621601 shutdown_connections 2024-09-19T04:05:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 --2- 172.21.15.19:0/3110621601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bd4106f10 0x7f4bd410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 --2- 172.21.15.19:0/3110621601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4102790 0x7f4bd41067e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 --2- 172.21.15.19:0/3110621601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd4101de0 0x7f4bd41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3110621601 >> 172.21.15.19:0/3110621601 conn(0x7f4bd40fd8c0 msgr2=0x7f4bd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3110621601 shutdown_connections 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.136+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3110621601 wait complete. 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bdbf8e640 1 Processor -- start 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bdbf8e640 1 -- start start 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bdbf8e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4101de0 0x7f4bd40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bdbf8e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd4102790 0x7f4bd4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bdbf8e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bd4106f10 0x7f4bd4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bdbf8e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4111fc0 con 0x7f4bd4101de0 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bdbf8e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4111e40 con 0x7f4bd4106f10 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bdbf8e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4112140 con 0x7f4bd4102790 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bd9d2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4101de0 0x7f4bd40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bd9d2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4101de0 0x7f4bd40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37966/0 (socket says 172.21.15.19:37966) 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bd9d2a640 1 -- 172.21.15.19:0/3161011291 learned_addr learned my addr 172.21.15.19:0/3161011291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.137+0000 7f4bd9529640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd4102790 0x7f4bd4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bda52b640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bd4106f10 0x7f4bd4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bd9d2a640 1 -- 172.21.15.19:0/3161011291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd4102790 msgr2=0x7f4bd4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bd9d2a640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd4102790 0x7f4bd4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bd9d2a640 1 -- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bd4106f10 msgr2=0x7f4bd4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bd9d2a640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bd4106f10 0x7f4bd4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bd9d2a640 1 -- 172.21.15.19:0/3161011291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd41aae10 con 0x7f4bd4101de0 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bda52b640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bd4106f10 0x7f4bd4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bd9d2a640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4101de0 0x7f4bd40744f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c970 tx=0x7f4bd000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0007bb0 con 0x7f4bd4101de0 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd0007d50 con 0x7f4bd4101de0 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd41aafe0 con 0x7f4bd4101de0 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000f450 con 0x7f4bd4101de0 2024-09-19T04:05:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.138+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd41ab440 con 0x7f4bd4101de0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.139+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bd4062900 con 0x7f4bd4101de0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.140+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4bd0005020 con 0x7f4bd4101de0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.141+0000 7f4bcaffd640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bc00a3350 0x7f4bc00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.141+0000 7f4bd9529640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bc00a3350 0x7f4bc00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.141+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f4bd00cfbf0 con 0x7f4bd4101de0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.141+0000 7f4bcaffd640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] conn(0x7f4bc00ae4a0 0x7f4bc00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.141+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 --> [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bd0004490 con 0x7f4bc00ae4a0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.141+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4bd00d42a0 con 0x7f4bd4101de0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.141+0000 7f4bd9529640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bc00a3350 0x7f4bc00a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006010 tx=0x7f4bc4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.142+0000 7f4bda52b640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] conn(0x7f4bc00ae4a0 0x7f4bc00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.142+0000 7f4bda52b640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] conn(0x7f4bc00ae4a0 0x7f4bc00b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.143+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 <== osd.1 v2:172.21.15.111:6800/3201221054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bd0004490 con 0x7f4bc00ae4a0 2024-09-19T04:05:50.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.157+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 --> [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bd4062d80 con 0x7f4bc00ae4a0 2024-09-19T04:05:50.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.157+0000 7f4bcaffd640 1 -- 172.21.15.19:0/3161011291 <== osd.1 v2:172.21.15.111:6800/3201221054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4bd4062d80 con 0x7f4bc00ae4a0 2024-09-19T04:05:50.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] conn(0x7f4bc00ae4a0 msgr2=0x7f4bc00b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] conn(0x7f4bc00ae4a0 0x7f4bc00b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bc00a3350 msgr2=0x7f4bc00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bc00a3350 0x7f4bc00a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006010 tx=0x7f4bc4000bc0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4101de0 msgr2=0x7f4bd40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 --2- 172.21.15.19:0/3161011291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bd4101de0 0x7f4bd40744f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c970 tx=0x7f4bd000ce30 comp rx=0 tx=0).stop 2024-09-19T04:05:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bda52b640 1 -- 172.21.15.19:0/3161011291 reap_dead start 2024-09-19T04:05:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 shutdown_connections 2024-09-19T04:05:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 >> 172.21.15.19:0/3161011291 conn(0x7f4bd40fd8c0 msgr2=0x7f4bd40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 shutdown_connections 2024-09-19T04:05:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.158+0000 7f4bdbf8e640 1 -- 172.21.15.19:0/3161011291 wait complete. 2024-09-19T04:05:50.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 -- 172.21.15.19:0/1536094854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c100a80 msgr2=0x7f084c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 --2- 172.21.15.19:0/1536094854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c100a80 0x7f084c100ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f083c009930 tx=0x7f083c02a420 comp rx=0 tx=0).stop 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 -- 172.21.15.19:0/1536094854 shutdown_connections 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 --2- 172.21.15.19:0/1536094854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f084c101420 0x7f084c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 --2- 172.21.15.19:0/1536094854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c100a80 0x7f084c100ee0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 --2- 172.21.15.19:0/1536094854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c106a80 0x7f084c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 -- 172.21.15.19:0/1536094854 >> 172.21.15.19:0/1536094854 conn(0x7f084c0fc7a0 msgr2=0x7f084c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 -- 172.21.15.19:0/1536094854 shutdown_connections 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 -- 172.21.15.19:0/1536094854 wait complete. 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 Processor -- start 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.397+0000 7f0853502640 1 -- start start 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0853502640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c100a80 0x7f084c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0853502640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f084c101420 0x7f084c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0853502640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c106a80 0x7f084c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0853502640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f084c110e80 con 0x7f084c106a80 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0853502640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f084c110d00 con 0x7f084c100a80 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0853502640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f084c111000 con 0x7f084c101420 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c106a80 0x7f084c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c106a80 0x7f084c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37902/0 (socket says 172.21.15.19:37902) 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 -- 172.21.15.19:0/3794912803 learned_addr learned my addr 172.21.15.19:0/3794912803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0850a9d640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f084c101420 0x7f084c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f085129e640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c100a80 0x7f084c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 -- 172.21.15.19:0/3794912803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f084c101420 msgr2=0x7f084c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f084c101420 0x7f084c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 -- 172.21.15.19:0/3794912803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c100a80 msgr2=0x7f084c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c100a80 0x7f084c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 -- 172.21.15.19:0/3794912803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c19d400 con 0x7f084c106a80 2024-09-19T04:05:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.398+0000 7f0851a9f640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c106a80 0x7f084c19cd20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f084400d750 tx=0x7f084400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.399+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844017070 con 0x7f084c106a80 2024-09-19T04:05:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.399+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0844005500 con 0x7f084c106a80 2024-09-19T04:05:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.399+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c19d6f0 con 0x7f084c106a80 2024-09-19T04:05:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.399+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844004d90 con 0x7f084c106a80 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.399+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1a4fd0 con 0x7f084c106a80 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.399+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f084c0fd030 con 0x7f084c106a80 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.401+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0844004f30 con 0x7f084c106a80 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.401+0000 7f08427fc640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f08380a3350 0x7f08380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.402+0000 7f085129e640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f08380a3350 0x7f08380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.402+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f08440cfab0 con 0x7f084c106a80 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.402+0000 7f08427fc640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] conn(0x7f08380ae490 0x7f08380b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.402+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 --> [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0844022e50 con 0x7f08380ae490 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.402+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f08440cfe60 con 0x7f084c106a80 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.402+0000 7f085129e640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f08380a3350 0x7f08380a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0848005fd0 tx=0x7f0848009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.402+0000 7f0850a9d640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] conn(0x7f08380ae490 0x7f08380b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.403+0000 7f0850a9d640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] conn(0x7f08380ae490 0x7f08380b0930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.403+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 <== osd.2 v2:172.21.15.196:6800/4085056128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0844022e50 con 0x7f08380ae490 2024-09-19T04:05:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.419+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 --> [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f084c1021c0 con 0x7f08380ae490 2024-09-19T04:05:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.419+0000 7f08427fc640 1 -- 172.21.15.19:0/3794912803 <== osd.2 v2:172.21.15.196:6800/4085056128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f084c1021c0 con 0x7f08380ae490 2024-09-19T04:05:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 >> [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] conn(0x7f08380ae490 msgr2=0x7f08380b0930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] conn(0x7f08380ae490 0x7f08380b0930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f08380a3350 msgr2=0x7f08380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f08380a3350 0x7f08380a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0848005fd0 tx=0x7f0848009680 comp rx=0 tx=0).stop 2024-09-19T04:05:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c106a80 msgr2=0x7f084c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 --2- 172.21.15.19:0/3794912803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f084c106a80 0x7f084c19cd20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f084400d750 tx=0x7f084400dc10 comp rx=0 tx=0).stop 2024-09-19T04:05:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0851a9f640 1 -- 172.21.15.19:0/3794912803 reap_dead start 2024-09-19T04:05:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 shutdown_connections 2024-09-19T04:05:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 >> 172.21.15.19:0/3794912803 conn(0x7f084c0fc7a0 msgr2=0x7f084c1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 shutdown_connections 2024-09-19T04:05:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:49.420+0000 7f0853502640 1 -- 172.21.15.19:0/3794912803 wait complete. 2024-09-19T04:05:50.890 INFO:teuthology.orchestra.run.smithi019.stdout:77309411401 2024-09-19T04:05:50.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T04:05:51.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:51 smithi019 ceph-mon[30938]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 900 B/s wr, 10 op/s; 6.5 KiB/s, 1 objects/s recovering 2024-09-19T04:05:51.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 -- 172.21.15.19:0/3068192253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 msgr2=0x7f6bc01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:51.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 --2- 172.21.15.19:0/3068192253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 0x7f6bc01029b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009900 tx=0x7f6bb402a480 comp rx=0 tx=0).stop 2024-09-19T04:05:51.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 -- 172.21.15.19:0/3068192253 shutdown_connections 2024-09-19T04:05:51.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 --2- 172.21.15.19:0/3068192253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc0102ef0 0x7f6bc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 --2- 172.21.15.19:0/3068192253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 0x7f6bc01029b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 --2- 172.21.15.19:0/3068192253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bc0100e20 0x7f6bc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 -- 172.21.15.19:0/3068192253 >> 172.21.15.19:0/3068192253 conn(0x7f6bc00fc7d0 msgr2=0x7f6bc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 -- 172.21.15.19:0/3068192253 shutdown_connections 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.025+0000 7f6bc5210640 1 -- 172.21.15.19:0/3068192253 wait complete. 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.026+0000 7f6bc5210640 1 Processor -- start 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.026+0000 7f6bc5210640 1 -- start start 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bc5210640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bc0100e20 0x7f6bc0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bc5210640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 0x7f6bc0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bc5210640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc0102ef0 0x7f6bc019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bc5210640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0110f80 con 0x7f6bc0102570 2024-09-19T04:05:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bc5210640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0110e00 con 0x7f6bc0102ef0 2024-09-19T04:05:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bc5210640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0111100 con 0x7f6bc0100e20 2024-09-19T04:05:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 0x7f6bc0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 0x7f6bc0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37938/0 (socket says 172.21.15.19:37938) 2024-09-19T04:05:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 -- 172.21.15.19:0/801353419 learned_addr learned my addr 172.21.15.19:0/801353419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbf59e640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc0102ef0 0x7f6bc019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbed9d640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bc0100e20 0x7f6bc0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 -- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bc0100e20 msgr2=0x7f6bc0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bc0100e20 0x7f6bc0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 -- 172.21.15.19:0/801353419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc0102ef0 msgr2=0x7f6bc019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc0102ef0 0x7f6bc019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 -- 172.21.15.19:0/801353419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc019d500 con 0x7f6bc0102570 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbed9d640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bc0100e20 0x7f6bc0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6bbe59c640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 0x7f6bc0198a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4006010 tx=0x7f6bb4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.027+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb403f070 con 0x7f6bc0102570 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.028+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc019d790 con 0x7f6bc0102570 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.028+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb4031c30 con 0x7f6bc0102570 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.028+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4035030 con 0x7f6bc0102570 2024-09-19T04:05:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.028+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc01a50d0 con 0x7f6bc0102570 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.029+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bc0062980 con 0x7f6bc0102570 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.030+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6bb4042070 con 0x7f6bc0102570 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.031+0000 7f6ba7fff640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6ba80a32d0 0x7f6ba80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.031+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f6bb40f0e60 con 0x7f6bc0102570 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.031+0000 7f6bbed9d640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6ba80a32d0 0x7f6ba80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.031+0000 7f6ba7fff640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] conn(0x7f6ba80ae420 0x7f6ba80b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.031+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 --> [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ba80b0f90 con 0x7f6ba80ae420 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.031+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6bb40f12b0 con 0x7f6bc0102570 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.031+0000 7f6bbed9d640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6ba80a32d0 0x7f6ba80a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0005fd0 tx=0x7f6bb0009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.031+0000 7f6bbf59e640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] conn(0x7f6ba80ae420 0x7f6ba80b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.032+0000 7f6bbf59e640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] conn(0x7f6ba80ae420 0x7f6ba80b08c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.033+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 <== osd.11 v2:172.21.15.196:6824/301665031 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ba80b0f90 con 0x7f6ba80ae420 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.046+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 --> [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bc0062d80 con 0x7f6ba80ae420 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6ba7fff640 1 -- 172.21.15.19:0/801353419 <== osd.11 v2:172.21.15.196:6824/301665031 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bc0062d80 con 0x7f6ba80ae420 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] conn(0x7f6ba80ae420 msgr2=0x7f6ba80b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bc5210640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] conn(0x7f6ba80ae420 0x7f6ba80b08c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6ba80a32d0 msgr2=0x7f6ba80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bc5210640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6ba80a32d0 0x7f6ba80a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0005fd0 tx=0x7f6bb0009520 comp rx=0 tx=0).stop 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 msgr2=0x7f6bc0198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bc5210640 1 --2- 172.21.15.19:0/801353419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bc0102570 0x7f6bc0198a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4006010 tx=0x7f6bb4003c10 comp rx=0 tx=0).stop 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bbf59e640 1 -- 172.21.15.19:0/801353419 reap_dead start 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 shutdown_connections 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.047+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 >> 172.21.15.19:0/801353419 conn(0x7f6bc00fc7d0 msgr2=0x7f6bc006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.048+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 shutdown_connections 2024-09-19T04:05:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:50.048+0000 7f6bc5210640 1 -- 172.21.15.19:0/801353419 wait complete. 2024-09-19T04:05:51.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:51 smithi111 ceph-mon[35365]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 900 B/s wr, 10 op/s; 6.5 KiB/s, 1 objects/s recovering 2024-09-19T04:05:51.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:51 smithi196 ceph-mon[35731]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 900 B/s wr, 10 op/s; 6.5 KiB/s, 1 objects/s recovering 2024-09-19T04:05:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.384+0000 7fe8205fc640 1 -- 172.21.15.19:0/3015683411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818101de0 msgr2=0x7fe8181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.384+0000 7fe8205fc640 1 --2- 172.21.15.19:0/3015683411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818101de0 0x7fe8181021c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe808009ca0 tx=0x7fe80802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.384+0000 7fe8205fc640 1 -- 172.21.15.19:0/3015683411 shutdown_connections 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 --2- 172.21.15.19:0/3015683411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe818106f10 0x7fe81810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 --2- 172.21.15.19:0/3015683411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe818102790 0x7fe8181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 --2- 172.21.15.19:0/3015683411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818101de0 0x7fe8181021c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 -- 172.21.15.19:0/3015683411 >> 172.21.15.19:0/3015683411 conn(0x7fe8180fd8c0 msgr2=0x7fe8180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 -- 172.21.15.19:0/3015683411 shutdown_connections 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 -- 172.21.15.19:0/3015683411 wait complete. 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 Processor -- start 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 -- start start 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe818101de0 0x7fe8181984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe818102790 0x7fe8181989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818106f10 0x7fe81819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe818111f90 con 0x7fe818106f10 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe818111e10 con 0x7fe818101de0 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.385+0000 7fe8205fc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe818112110 con 0x7fe818102790 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818106f10 0x7fe81819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818106f10 0x7fe81819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42134/0 (socket says 172.21.15.19:42134) 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 -- 172.21.15.19:0/2448251290 learned_addr learned my addr 172.21.15.19:0/2448251290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81e398640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe818101de0 0x7fe8181984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81db97640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe818102790 0x7fe8181989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 -- 172.21.15.19:0/2448251290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe818102790 msgr2=0x7fe8181989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe818102790 0x7fe8181989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 -- 172.21.15.19:0/2448251290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe818101de0 msgr2=0x7fe8181984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe818101de0 0x7fe8181984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 -- 172.21.15.19:0/2448251290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81819d460 con 0x7fe818106f10 2024-09-19T04:05:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe81eb99640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818106f10 0x7fe81819cd80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe810009fd0 tx=0x7fe81000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810011830 con 0x7fe818106f10 2024-09-19T04:05:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe810016070 con 0x7fe818106f10 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81819d750 con 0x7fe818106f10 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81000f2e0 con 0x7fe818106f10 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.386+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8181a5030 con 0x7fe818106f10 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.387+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe818062980 con 0x7fe818106f10 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.389+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe8100119d0 con 0x7fe818106f10 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.389+0000 7fe8177fe640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8000a3350 0x7fe8000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.390+0000 7fe81e398640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8000a3350 0x7fe8000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.390+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fe8100cfe00 con 0x7fe818106f10 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.390+0000 7fe8177fe640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] conn(0x7fe8000ae420 0x7fe8000b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.390+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 --> [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe81001ee50 con 0x7fe8000ae420 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.390+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe8100d0250 con 0x7fe818106f10 2024-09-19T04:05:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.390+0000 7fe81e398640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8000a3350 0x7fe8000a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe808006010 tx=0x7fe808009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.390+0000 7fe81db97640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] conn(0x7fe8000ae420 0x7fe8000b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.390+0000 7fe81db97640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] conn(0x7fe8000ae420 0x7fe8000b08c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.391+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 <== osd.10 v2:172.21.15.111:6824/610848665 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe81001ee50 con 0x7fe8000ae420 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.405+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 --> [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe818062e00 con 0x7fe8000ae420 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.405+0000 7fe8177fe640 1 -- 172.21.15.19:0/2448251290 <== osd.10 v2:172.21.15.111:6824/610848665 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe818062e00 con 0x7fe8000ae420 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 >> [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] conn(0x7fe8000ae420 msgr2=0x7fe8000b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] conn(0x7fe8000ae420 0x7fe8000b08c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8000a3350 msgr2=0x7fe8000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe8000a3350 0x7fe8000a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe808006010 tx=0x7fe808009360 comp rx=0 tx=0).stop 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818106f10 msgr2=0x7fe81819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 --2- 172.21.15.19:0/2448251290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818106f10 0x7fe81819cd80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe810009fd0 tx=0x7fe81000b8e0 comp rx=0 tx=0).stop 2024-09-19T04:05:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe81eb99640 1 -- 172.21.15.19:0/2448251290 reap_dead start 2024-09-19T04:05:52.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 shutdown_connections 2024-09-19T04:05:52.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 >> 172.21.15.19:0/2448251290 conn(0x7fe8180fd8c0 msgr2=0x7fe8180fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:52.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 shutdown_connections 2024-09-19T04:05:52.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:51.406+0000 7fe8205fc640 1 -- 172.21.15.19:0/2448251290 wait complete. 2024-09-19T04:05:53.557 INFO:teuthology.orchestra.run.smithi019.stdout:300647710752 2024-09-19T04:05:53.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T04:05:53.724 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:53 smithi019 ceph-mon[30938]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 811 B/s wr, 11 op/s; 5.9 KiB/s, 0 objects/s recovering 2024-09-19T04:05:53.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:53 smithi111 ceph-mon[35365]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 811 B/s wr, 11 op/s; 5.9 KiB/s, 0 objects/s recovering 2024-09-19T04:05:53.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 -- 172.21.15.19:0/3215862357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704101ba0 msgr2=0x7ff70410d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:53.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 --2- 172.21.15.19:0/3215862357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704101ba0 0x7ff70410d970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009ca0 tx=0x7ff6fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 -- 172.21.15.19:0/3215862357 shutdown_connections 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 --2- 172.21.15.19:0/3215862357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704101ba0 0x7ff70410d970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 --2- 172.21.15.19:0/3215862357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704101200 0x7ff704101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 --2- 172.21.15.19:0/3215862357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff704105ad0 0x7ff704105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 -- 172.21.15.19:0/3215862357 >> 172.21.15.19:0/3215862357 conn(0x7ff704077210 msgr2=0x7ff7040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 -- 172.21.15.19:0/3215862357 shutdown_connections 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 -- 172.21.15.19:0/3215862357 wait complete. 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.931+0000 7ff708a84640 1 Processor -- start 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff708a84640 1 -- start start 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff708a84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704101200 0x7ff70419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff708a84640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff704101ba0 0x7ff70419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff708a84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704105ad0 0x7ff70419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff708a84640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff704110ef0 con 0x7ff704105ad0 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff708a84640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff704110d70 con 0x7ff704101ba0 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff708a84640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff704111070 con 0x7ff704101200 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff702d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704105ad0 0x7ff70419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff702d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704105ad0 0x7ff70419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42172/0 (socket says 172.21.15.19:42172) 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff702d9d640 1 -- 172.21.15.19:0/228979130 learned_addr learned my addr 172.21.15.19:0/228979130 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff70259c640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704101200 0x7ff70419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff701d9b640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff704101ba0 0x7ff70419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff702d9d640 1 -- 172.21.15.19:0/228979130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704101200 msgr2=0x7ff70419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff702d9d640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704101200 0x7ff70419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff702d9d640 1 -- 172.21.15.19:0/228979130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff704101ba0 msgr2=0x7ff70419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff702d9d640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff704101ba0 0x7ff70419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.932+0000 7ff702d9d640 1 -- 172.21.15.19:0/228979130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70419f4f0 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.933+0000 7ff702d9d640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704105ad0 0x7ff70419ee10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc003f40 tx=0x7ff6fc003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.933+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc031790 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.933+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc031930 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.933+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70419f780 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.933+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc030bf0 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.933+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7041a70c0 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.934+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff704062980 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.935+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff6fc004040 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.936+0000 7ff6f37fe640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff6ec0a32d0 0x7ff6ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.936+0000 7ff70259c640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff6ec0a32d0 0x7ff6ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.936+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7ff6fc0f1470 con 0x7ff704105ad0 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.936+0000 7ff6f37fe640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] conn(0x7ff6ec0ae420 0x7ff6ec0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.936+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 --> [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6ec0b0f90 con 0x7ff6ec0ae420 2024-09-19T04:05:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.936+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc0f18c0 con 0x7ff704105ad0 2024-09-19T04:05:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.936+0000 7ff701d9b640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] conn(0x7ff6ec0ae420 0x7ff6ec0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.936+0000 7ff70259c640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff6ec0a32d0 0x7ff6ec0a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4005d00 tx=0x7ff6f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.937+0000 7ff701d9b640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] conn(0x7ff6ec0ae420 0x7ff6ec0b08c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.937+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 <== osd.6 v2:172.21.15.19:6818/3521642658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6ec0b0f90 con 0x7ff6ec0ae420 2024-09-19T04:05:53.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.951+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 --> [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff704062e00 con 0x7ff6ec0ae420 2024-09-19T04:05:53.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.951+0000 7ff6f37fe640 1 -- 172.21.15.19:0/228979130 <== osd.6 v2:172.21.15.19:6818/3521642658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff704062e00 con 0x7ff6ec0ae420 2024-09-19T04:05:53.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.951+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] conn(0x7ff6ec0ae420 msgr2=0x7ff6ec0b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:53.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.951+0000 7ff708a84640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] conn(0x7ff6ec0ae420 0x7ff6ec0b08c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:53.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff6ec0a32d0 msgr2=0x7ff6ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff708a84640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff6ec0a32d0 0x7ff6ec0a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4005d00 tx=0x7ff6f400b410 comp rx=0 tx=0).stop 2024-09-19T04:05:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704105ad0 msgr2=0x7ff70419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff708a84640 1 --2- 172.21.15.19:0/228979130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff704105ad0 0x7ff70419ee10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc003f40 tx=0x7ff6fc003f70 comp rx=0 tx=0).stop 2024-09-19T04:05:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff702d9d640 1 -- 172.21.15.19:0/228979130 reap_dead start 2024-09-19T04:05:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 shutdown_connections 2024-09-19T04:05:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 >> 172.21.15.19:0/228979130 conn(0x7ff704077210 msgr2=0x7ff704104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 shutdown_connections 2024-09-19T04:05:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.952+0000 7ff708a84640 1 -- 172.21.15.19:0/228979130 wait complete. 2024-09-19T04:05:53.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:53 smithi196 ceph-mon[35731]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 811 B/s wr, 11 op/s; 5.9 KiB/s, 0 objects/s recovering 2024-09-19T04:05:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.415+0000 7fcc2f77e640 1 -- 172.21.15.19:0/3923162211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 msgr2=0x7fcc2810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 --2- 172.21.15.19:0/3923162211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 0x7fcc2810da40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009ca0 tx=0x7fcc2002a950 comp rx=0 tx=0).stop 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 -- 172.21.15.19:0/3923162211 shutdown_connections 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 --2- 172.21.15.19:0/3923162211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 0x7fcc2810da40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 --2- 172.21.15.19:0/3923162211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc28102570 0x7fcc281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 --2- 172.21.15.19:0/3923162211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc28100e20 0x7fcc28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 -- 172.21.15.19:0/3923162211 >> 172.21.15.19:0/3923162211 conn(0x7fcc280fc7d0 msgr2=0x7fcc280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 -- 172.21.15.19:0/3923162211 shutdown_connections 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 -- 172.21.15.19:0/3923162211 wait complete. 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 Processor -- start 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.416+0000 7fcc2f77e640 1 -- start start 2024-09-19T04:05:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2f77e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc28100e20 0x7fcc28198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2f77e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc28102570 0x7fcc281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2f77e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 0x7fcc2819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2f77e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281120a0 con 0x7fcc28102ef0 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2f77e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28111f20 con 0x7fcc28100e20 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2f77e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28112220 con 0x7fcc28102570 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 0x7fcc2819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2cd19640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc28102570 0x7fcc281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 0x7fcc2819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42186/0 (socket says 172.21.15.19:42186) 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 -- 172.21.15.19:0/283369273 learned_addr learned my addr 172.21.15.19:0/283369273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2d51a640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc28100e20 0x7fcc28198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 -- 172.21.15.19:0/283369273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc28102570 msgr2=0x7fcc281989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc28102570 0x7fcc281989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 -- 172.21.15.19:0/283369273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc28100e20 msgr2=0x7fcc28198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc28100e20 0x7fcc28198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 -- 172.21.15.19:0/283369273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2819d410 con 0x7fcc28102ef0 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.417+0000 7fcc2dd1b640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 0x7fcc2819cd30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc2002ae70 tx=0x7fcc2002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.418+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20034040 con 0x7fcc28102ef0 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.418+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2819d6a0 con 0x7fcc28102ef0 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.418+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc20030920 con 0x7fcc28102ef0 2024-09-19T04:05:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.418+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20045480 con 0x7fcc28102ef0 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.418+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc281a4fe0 con 0x7fcc28102ef0 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.419+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc28107300 con 0x7fcc28102ef0 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.420+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcc20037020 con 0x7fcc28102ef0 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.421+0000 7fcc267fc640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc100a32d0 0x7fcc100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.421+0000 7fcc2d51a640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc100a32d0 0x7fcc100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.421+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fcc200f0f50 con 0x7fcc28102ef0 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.421+0000 7fcc267fc640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] conn(0x7fcc100ae420 0x7fcc100b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.421+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 --> [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc20050e50 con 0x7fcc100ae420 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.421+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fcc200f1300 con 0x7fcc28102ef0 2024-09-19T04:05:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.421+0000 7fcc2d51a640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc100a32d0 0x7fcc100a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800bbe0 tx=0x7fcc1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.421+0000 7fcc2cd19640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] conn(0x7fcc100ae420 0x7fcc100b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.422+0000 7fcc2cd19640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] conn(0x7fcc100ae420 0x7fcc100b08c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.422+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 <== osd.8 v2:172.21.15.196:6816/189709907 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc20050e50 con 0x7fcc100ae420 2024-09-19T04:05:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.437+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 --> [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc281021b0 con 0x7fcc100ae420 2024-09-19T04:05:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc267fc640 1 -- 172.21.15.19:0/283369273 <== osd.8 v2:172.21.15.196:6816/189709907 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc281021b0 con 0x7fcc100ae420 2024-09-19T04:05:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 >> [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] conn(0x7fcc100ae420 msgr2=0x7fcc100b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] conn(0x7fcc100ae420 0x7fcc100b08c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc100a32d0 msgr2=0x7fcc100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc100a32d0 0x7fcc100a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800bbe0 tx=0x7fcc1800b390 comp rx=0 tx=0).stop 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 msgr2=0x7fcc2819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 --2- 172.21.15.19:0/283369273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc28102ef0 0x7fcc2819cd30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc2002ae70 tx=0x7fcc2002f5d0 comp rx=0 tx=0).stop 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2dd1b640 1 -- 172.21.15.19:0/283369273 reap_dead start 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 shutdown_connections 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 >> 172.21.15.19:0/283369273 conn(0x7fcc280fc7d0 msgr2=0x7fcc281068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.438+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 shutdown_connections 2024-09-19T04:05:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:54.439+0000 7fcc2f77e640 1 -- 172.21.15.19:0/283369273 wait complete. 2024-09-19T04:05:54.984 INFO:teuthology.orchestra.run.smithi019.stdout:214748364847 2024-09-19T04:05:54.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T04:05:55.385 INFO:teuthology.orchestra.run.smithi019.stdout:137438953535 2024-09-19T04:05:55.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T04:05:55.525 INFO:teuthology.orchestra.run.smithi019.stdout:330712481820 2024-09-19T04:05:55.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T04:05:55.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:55 smithi019 ceph-mon[30938]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-19T04:05:55.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 -- 172.21.15.19:0/3432385244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 msgr2=0x7fdde41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 --2- 172.21.15.19:0/3432385244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 0x7fdde41067e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009900 tx=0x7fddd002a410 comp rx=0 tx=0).stop 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 -- 172.21.15.19:0/3432385244 shutdown_connections 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 --2- 172.21.15.19:0/3432385244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4106f10 0x7fdde410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 --2- 172.21.15.19:0/3432385244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 0x7fdde41067e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 --2- 172.21.15.19:0/3432385244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde4101de0 0x7fdde41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 -- 172.21.15.19:0/3432385244 >> 172.21.15.19:0/3432385244 conn(0x7fdde40fd8c0 msgr2=0x7fdde40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 -- 172.21.15.19:0/3432385244 shutdown_connections 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.792+0000 7fddeaec5640 1 -- 172.21.15.19:0/3432385244 wait complete. 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fddeaec5640 1 Processor -- start 2024-09-19T04:05:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fddeaec5640 1 -- start start 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fddeaec5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde4101de0 0x7fdde40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fddeaec5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 0x7fdde4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fddeaec5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4106f10 0x7fdde4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fddeaec5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4112000 con 0x7fdde4102790 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fddeaec5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4111e80 con 0x7fdde4106f10 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fddeaec5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4112180 con 0x7fdde4101de0 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fdde3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 0x7fdde4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fdde3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 0x7fdde4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42204/0 (socket says 172.21.15.19:42204) 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fdde3fff640 1 -- 172.21.15.19:0/3985768110 learned_addr learned my addr 172.21.15.19:0/3985768110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fdde8c61640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde4101de0 0x7fdde40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.793+0000 7fdde9462640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4106f10 0x7fdde4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde3fff640 1 -- 172.21.15.19:0/3985768110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde4101de0 msgr2=0x7fdde40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde3fff640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde4101de0 0x7fdde40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:55.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde3fff640 1 -- 172.21.15.19:0/3985768110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4106f10 msgr2=0x7fdde4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde3fff640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde4106f10 0x7fdde4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde3fff640 1 -- 172.21.15.19:0/3985768110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde41a6990 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde3fff640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 0x7fdde4074a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fddd0004770 tx=0x7fddd0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd003f070 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd0031ca0 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0035030 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde41a6c20 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.794+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a7150 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.795+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdde4062880 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.796+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fddd0042070 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.797+0000 7fdde1ffb640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddd40a3350 0x7fddd40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.797+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fddd00f0a00 con 0x7fdde4102790 2024-09-19T04:05:55.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.797+0000 7fdde8c61640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddd40a3350 0x7fddd40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:55.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.797+0000 7fdde1ffb640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] conn(0x7fddd40ae4a0 0x7fddd40b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:55.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.797+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 --> [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddd0038e90 con 0x7fddd40ae4a0 2024-09-19T04:05:55.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.797+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fddd00f0db0 con 0x7fdde4102790 2024-09-19T04:05:55.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.798+0000 7fdde9462640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] conn(0x7fddd40ae4a0 0x7fddd40b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:55.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.798+0000 7fdde8c61640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddd40a3350 0x7fddd40a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005fd0 tx=0x7fddd8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:55.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.798+0000 7fdde9462640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] conn(0x7fddd40ae4a0 0x7fddd40b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:55.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.799+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 <== osd.9 v2:172.21.15.19:6826/2115247228 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddd0038e90 con 0x7fddd40ae4a0 2024-09-19T04:05:55.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.813+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 --> [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdde4062c80 con 0x7fddd40ae4a0 2024-09-19T04:05:55.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.813+0000 7fdde1ffb640 1 -- 172.21.15.19:0/3985768110 <== osd.9 v2:172.21.15.19:6826/2115247228 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdde4062c80 con 0x7fddd40ae4a0 2024-09-19T04:05:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.813+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] conn(0x7fddd40ae4a0 msgr2=0x7fddd40b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.813+0000 7fddeaec5640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] conn(0x7fddd40ae4a0 0x7fddd40b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.813+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddd40a3350 msgr2=0x7fddd40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.813+0000 7fddeaec5640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fddd40a3350 0x7fddd40a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005fd0 tx=0x7fddd8005850 comp rx=0 tx=0).stop 2024-09-19T04:05:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.813+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 msgr2=0x7fdde4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.813+0000 7fddeaec5640 1 --2- 172.21.15.19:0/3985768110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde4102790 0x7fdde4074a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fddd0004770 tx=0x7fddd0003c10 comp rx=0 tx=0).stop 2024-09-19T04:05:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.814+0000 7fdde9462640 1 -- 172.21.15.19:0/3985768110 reap_dead start 2024-09-19T04:05:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.814+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 shutdown_connections 2024-09-19T04:05:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.814+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 >> 172.21.15.19:0/3985768110 conn(0x7fdde40fd8c0 msgr2=0x7fdde40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.814+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 shutdown_connections 2024-09-19T04:05:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:55.814+0000 7fddeaec5640 1 -- 172.21.15.19:0/3985768110 wait complete. 2024-09-19T04:05:55.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:55 smithi111 ceph-mon[35365]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-19T04:05:55.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:55 smithi196 ceph-mon[35731]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-19T04:05:56.674 INFO:teuthology.orchestra.run.smithi019.stdout:193273528374 2024-09-19T04:05:56.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T04:05:57.577 INFO:teuthology.orchestra.run.smithi019.stdout:55834574927 2024-09-19T04:05:57.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T04:05:57.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.722+0000 7fca32615640 1 -- 172.21.15.19:0/4098532124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2c100e20 msgr2=0x7fca2c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:57.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.722+0000 7fca32615640 1 --2- 172.21.15.19:0/4098532124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2c100e20 0x7fca2c101200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009900 tx=0x7fca1c02a480 comp rx=0 tx=0).stop 2024-09-19T04:05:57.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.722+0000 7fca32615640 1 -- 172.21.15.19:0/4098532124 shutdown_connections 2024-09-19T04:05:57.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.722+0000 7fca32615640 1 --2- 172.21.15.19:0/4098532124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca2c102ef0 0x7fca2c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:57.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.722+0000 7fca32615640 1 --2- 172.21.15.19:0/4098532124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c102570 0x7fca2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.722+0000 7fca32615640 1 --2- 172.21.15.19:0/4098532124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2c100e20 0x7fca2c101200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.722+0000 7fca32615640 1 -- 172.21.15.19:0/4098532124 >> 172.21.15.19:0/4098532124 conn(0x7fca2c0fc7d0 msgr2=0x7fca2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.723+0000 7fca32615640 1 -- 172.21.15.19:0/4098532124 shutdown_connections 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.723+0000 7fca32615640 1 -- 172.21.15.19:0/4098532124 wait complete. 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.723+0000 7fca32615640 1 Processor -- start 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca32615640 1 -- start start 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca32615640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca2c100e20 0x7fca2c197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca32615640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c102570 0x7fca2c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca32615640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2c102ef0 0x7fca2c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca32615640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c110fa0 con 0x7fca2c102ef0 2024-09-19T04:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca32615640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c110e20 con 0x7fca2c100e20 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca32615640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c111120 con 0x7fca2c102570 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca30bb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2c102ef0 0x7fca2c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca2bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca2c100e20 0x7fca2c197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca2bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca2c100e20 0x7fca2c197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:35964/0 (socket says 172.21.15.19:35964) 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca30bb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2c102ef0 0x7fca2c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42162/0 (socket says 172.21.15.19:42162) 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca2bfff640 1 -- 172.21.15.19:0/641736587 learned_addr learned my addr 172.21.15.19:0/641736587 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.724+0000 7fca2b7fe640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c102570 0x7fca2c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca2bfff640 1 -- 172.21.15.19:0/641736587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c102570 msgr2=0x7fca2c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca2bfff640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c102570 0x7fca2c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca2bfff640 1 -- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2c102ef0 msgr2=0x7fca2c19bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca2bfff640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2c102ef0 0x7fca2c19bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca2bfff640 1 -- 172.21.15.19:0/641736587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c19c3b0 con 0x7fca2c100e20 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca2bfff640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca2c100e20 0x7fca2c197400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca1c006010 tx=0x7fca1c003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c035030 con 0x7fca2c100e20 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c19c640 con 0x7fca2c100e20 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1c03f070 con 0x7fca2c100e20 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c069f70 con 0x7fca2c100e20 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.725+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c031da0 con 0x7fca2c100e20 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.726+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca2c1072c0 con 0x7fca2c100e20 2024-09-19T04:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.727+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fca1c03b970 con 0x7fca2c100e20 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.728+0000 7fca297fa640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fca140a32d0 0x7fca140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.728+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fca1c0f2ed0 con 0x7fca2c100e20 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.728+0000 7fca2b7fe640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fca140a32d0 0x7fca140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.728+0000 7fca297fa640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] conn(0x7fca140ae420 0x7fca140b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.728+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 --> [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca1c03eec0 con 0x7fca140ae420 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.728+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fca1c0f3280 con 0x7fca2c100e20 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.729+0000 7fca30bb2640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] conn(0x7fca140ae420 0x7fca140b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.729+0000 7fca2b7fe640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fca140a32d0 0x7fca140a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0ffd30 tx=0x7fca20009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.729+0000 7fca30bb2640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] conn(0x7fca140ae420 0x7fca140b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.730+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 <== osd.3 v2:172.21.15.19:6810/3319817620 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca1c03eec0 con 0x7fca140ae420 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.743+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 --> [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fca2c1021b0 con 0x7fca140ae420 2024-09-19T04:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca297fa640 1 -- 172.21.15.19:0/641736587 <== osd.3 v2:172.21.15.19:6810/3319817620 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fca2c1021b0 con 0x7fca140ae420 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] conn(0x7fca140ae420 msgr2=0x7fca140b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] conn(0x7fca140ae420 0x7fca140b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fca140a32d0 msgr2=0x7fca140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fca140a32d0 0x7fca140a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0ffd30 tx=0x7fca20009110 comp rx=0 tx=0).stop 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca2c100e20 msgr2=0x7fca2c197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 --2- 172.21.15.19:0/641736587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca2c100e20 0x7fca2c197400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca1c006010 tx=0x7fca1c003c80 comp rx=0 tx=0).stop 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca30bb2640 1 -- 172.21.15.19:0/641736587 reap_dead start 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 shutdown_connections 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 >> 172.21.15.19:0/641736587 conn(0x7fca2c0fc7d0 msgr2=0x7fca2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 shutdown_connections 2024-09-19T04:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:53.744+0000 7fca32615640 1 -- 172.21.15.19:0/641736587 wait complete. 2024-09-19T04:05:57.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:57 smithi019 ceph-mon[30938]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s; 1 B/s, 0 objects/s recovering 2024-09-19T04:05:57.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:57 smithi111 ceph-mon[35365]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s; 1 B/s, 0 objects/s recovering 2024-09-19T04:05:57.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:57 smithi196 ceph-mon[35731]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s; 1 B/s, 0 objects/s recovering 2024-09-19T04:05:58.400 INFO:teuthology.orchestra.run.smithi019.stdout:107374182469 2024-09-19T04:05:58.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T04:05:58.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 -- 172.21.15.19:0/2045261297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4106a80 msgr2=0x7fd2b4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:58.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 --2- 172.21.15.19:0/2045261297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4106a80 0x7fd2b4106e60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009930 tx=0x7fd2a402a420 comp rx=0 tx=0).stop 2024-09-19T04:05:58.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 -- 172.21.15.19:0/2045261297 shutdown_connections 2024-09-19T04:05:58.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 --2- 172.21.15.19:0/2045261297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b4101420 0x7fd2b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 --2- 172.21.15.19:0/2045261297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2b4100a80 0x7fd2b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 --2- 172.21.15.19:0/2045261297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4106a80 0x7fd2b4106e60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 -- 172.21.15.19:0/2045261297 >> 172.21.15.19:0/2045261297 conn(0x7fd2b40fc7a0 msgr2=0x7fd2b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 -- 172.21.15.19:0/2045261297 shutdown_connections 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 -- 172.21.15.19:0/2045261297 wait complete. 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 Processor -- start 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.706+0000 7fd2b9910640 1 -- start start 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b9910640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4100a80 0x7fd2b41985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b9910640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2b4101420 0x7fd2b4198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b9910640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b4106a80 0x7fd2b419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b9910640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b4112020 con 0x7fd2b4100a80 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b9910640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b4111ea0 con 0x7fd2b4101420 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b9910640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b41121a0 con 0x7fd2b4106a80 2024-09-19T04:05:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4100a80 0x7fd2b41985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4100a80 0x7fd2b41985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42224/0 (socket says 172.21.15.19:42224) 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/1630121801 learned_addr learned my addr 172.21.15.19:0/1630121801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b27fc640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2b4101420 0x7fd2b4198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b37fe640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b4106a80 0x7fd2b419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/1630121801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b4106a80 msgr2=0x7fd2b419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b4106a80 0x7fd2b419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/1630121801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2b4101420 msgr2=0x7fd2b4198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2b4101420 0x7fd2b4198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/1630121801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b419d590 con 0x7fd2b4100a80 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.707+0000 7fd2b2ffd640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4100a80 0x7fd2b41985e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009a50 tx=0x7fd2a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.708+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a403f030 con 0x7fd2b4100a80 2024-09-19T04:05:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.708+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4003ad0 con 0x7fd2b4100a80 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.708+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b419d820 con 0x7fd2b4100a80 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.708+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4035070 con 0x7fd2b4100a80 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.708+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b41a5160 con 0x7fd2b4100a80 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.709+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2b4062980 con 0x7fd2b4100a80 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.710+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd2a4042030 con 0x7fd2b4100a80 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.710+0000 7fd2b890e640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd29c0a3350 0x7fd29c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.711+0000 7fd2b27fc640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd29c0a3350 0x7fd29c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.711+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fd2a40f0980 con 0x7fd2b4100a80 2024-09-19T04:05:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.711+0000 7fd2b890e640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] conn(0x7fd29c0ae370 0x7fd29c0b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.711+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 --> [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd29c0b0ee0 con 0x7fd29c0ae370 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.711+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd2a40f0d30 con 0x7fd2b4100a80 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.711+0000 7fd2b27fc640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd29c0a3350 0x7fd29c0a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40fee00 tx=0x7fd2a8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.711+0000 7fd2b37fe640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] conn(0x7fd29c0ae370 0x7fd29c0b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.712+0000 7fd2b37fe640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] conn(0x7fd29c0ae370 0x7fd29c0b0810 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.712+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 <== osd.5 v2:172.21.15.196:6808/919577999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd29c0b0ee0 con 0x7fd29c0ae370 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.726+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 --> [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2b4062d80 con 0x7fd29c0ae370 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b890e640 1 -- 172.21.15.19:0/1630121801 <== osd.5 v2:172.21.15.196:6808/919577999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd2b4062d80 con 0x7fd29c0ae370 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 >> [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] conn(0x7fd29c0ae370 msgr2=0x7fd29c0b0810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:58.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] conn(0x7fd29c0ae370 0x7fd29c0b0810 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd29c0a3350 msgr2=0x7fd29c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd29c0a3350 0x7fd29c0a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40fee00 tx=0x7fd2a8009110 comp rx=0 tx=0).stop 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4100a80 msgr2=0x7fd2b41985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 --2- 172.21.15.19:0/1630121801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b4100a80 0x7fd2b41985e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009a50 tx=0x7fd2a4004270 comp rx=0 tx=0).stop 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b37fe640 1 -- 172.21.15.19:0/1630121801 reap_dead start 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 shutdown_connections 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 >> 172.21.15.19:0/1630121801 conn(0x7fd2b40fc7a0 msgr2=0x7fd2b40fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 shutdown_connections 2024-09-19T04:05:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:58.727+0000 7fd2b9910640 1 -- 172.21.15.19:0/1630121801 wait complete. 2024-09-19T04:05:59.430 INFO:teuthology.orchestra.run.smithi019.stdout:158913790011 2024-09-19T04:05:59.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T04:05:59.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:05:59 smithi019 ceph-mon[30938]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:05:59.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.704+0000 7fe5b8a59640 1 -- 172.21.15.19:0/3916573006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b4101280 msgr2=0x7fe5b410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 --2- 172.21.15.19:0/3916573006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b4101280 0x7fe5b410df40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009900 tx=0x7fe59c02a410 comp rx=0 tx=0).stop 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 -- 172.21.15.19:0/3916573006 shutdown_connections 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 --2- 172.21.15.19:0/3916573006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b410e480 0x7fe5b4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 --2- 172.21.15.19:0/3916573006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b4101280 0x7fe5b410df40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 --2- 172.21.15.19:0/3916573006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5b41008d0 0x7fe5b4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 -- 172.21.15.19:0/3916573006 >> 172.21.15.19:0/3916573006 conn(0x7fe5b40fc7a0 msgr2=0x7fe5b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 -- 172.21.15.19:0/3916573006 shutdown_connections 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 -- 172.21.15.19:0/3916573006 wait complete. 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.705+0000 7fe5b8a59640 1 Processor -- start 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b8a59640 1 -- start start 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b8a59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b41008d0 0x7fe5b419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b8a59640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5b4101280 0x7fe5b419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b8a59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b410e480 0x7fe5b419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b8a59640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41141a0 con 0x7fe5b41008d0 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b8a59640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4114020 con 0x7fe5b4101280 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b8a59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4114320 con 0x7fe5b410e480 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b41008d0 0x7fe5b419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b41008d0 0x7fe5b419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42246/0 (socket says 172.21.15.19:42246) 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b259c640 1 -- 172.21.15.19:0/164451121 learned_addr learned my addr 172.21.15.19:0/164451121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b2d9d640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b410e480 0x7fe5b419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b1d9b640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5b4101280 0x7fe5b419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b259c640 1 -- 172.21.15.19:0/164451121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b410e480 msgr2=0x7fe5b419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b259c640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b410e480 0x7fe5b419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b259c640 1 -- 172.21.15.19:0/164451121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5b4101280 msgr2=0x7fe5b419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b259c640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5b4101280 0x7fe5b419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b259c640 1 -- 172.21.15.19:0/164451121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b419f670 con 0x7fe5b41008d0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.706+0000 7fe5b1d9b640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5b4101280 0x7fe5b419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.707+0000 7fe5b259c640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b41008d0 0x7fe5b419a6c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400d950 tx=0x7fe5a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.707+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4017030 con 0x7fe5b41008d0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.707+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b419f960 con 0x7fe5b41008d0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.707+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a4004020 con 0x7fe5b41008d0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.707+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40047a0 con 0x7fe5b41008d0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.707+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b41a7240 con 0x7fe5b41008d0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.708+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5b4062980 con 0x7fe5b41008d0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.709+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe5a40041d0 con 0x7fe5b41008d0 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.710+0000 7fe5ab7fe640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe5a00a32d0 0x7fe5a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.710+0000 7fe5b1d9b640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe5a00a32d0 0x7fe5a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.710+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fe5a40cff60 con 0x7fe5b41008d0 2024-09-19T04:05:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.710+0000 7fe5ab7fe640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] conn(0x7fe5a00ae420 0x7fe5a00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:05:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.711+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 --> [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5a00b0f90 con 0x7fe5a00ae420 2024-09-19T04:05:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.711+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe5a40d0310 con 0x7fe5b41008d0 2024-09-19T04:05:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.711+0000 7fe5b1d9b640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe5a00a32d0 0x7fe5a00a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5b41102c0 tx=0x7fe59c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.711+0000 7fe5b2d9d640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] conn(0x7fe5a00ae420 0x7fe5a00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:05:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.711+0000 7fe5b2d9d640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] conn(0x7fe5a00ae420 0x7fe5a00b08c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:05:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.712+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 <== osd.0 v2:172.21.15.19:6802/2401652913 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5a00b0f90 con 0x7fe5a00ae420 2024-09-19T04:05:59.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 --> [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe5b4062d80 con 0x7fe5a00ae420 2024-09-19T04:05:59.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5ab7fe640 1 -- 172.21.15.19:0/164451121 <== osd.0 v2:172.21.15.19:6802/2401652913 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe5b4062d80 con 0x7fe5a00ae420 2024-09-19T04:05:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] conn(0x7fe5a00ae420 msgr2=0x7fe5a00b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] conn(0x7fe5a00ae420 0x7fe5a00b08c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:05:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe5a00a32d0 msgr2=0x7fe5a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe5a00a32d0 0x7fe5a00a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5b41102c0 tx=0x7fe59c005a20 comp rx=0 tx=0).stop 2024-09-19T04:05:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b41008d0 msgr2=0x7fe5b419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:05:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 --2- 172.21.15.19:0/164451121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b41008d0 0x7fe5b419a6c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400d950 tx=0x7fe5a400de10 comp rx=0 tx=0).stop 2024-09-19T04:05:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b2d9d640 1 -- 172.21.15.19:0/164451121 reap_dead start 2024-09-19T04:05:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 shutdown_connections 2024-09-19T04:05:59.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.726+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 >> 172.21.15.19:0/164451121 conn(0x7fe5b40fc7a0 msgr2=0x7fe5b40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:05:59.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.727+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 shutdown_connections 2024-09-19T04:05:59.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:05:59.727+0000 7fe5b8a59640 1 -- 172.21.15.19:0/164451121 wait complete. 2024-09-19T04:05:59.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:05:59 smithi111 ceph-mon[35365]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:05:59.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:05:59 smithi196 ceph-mon[35731]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:06:00.421 INFO:teuthology.orchestra.run.smithi019.stdout:279172874279 2024-09-19T04:06:00.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T04:06:00.898 INFO:teuthology.orchestra.run.smithi019.stdout:34359738454 2024-09-19T04:06:00.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T04:06:01.309 INFO:teuthology.orchestra.run.smithi019.stdout:244813135916 2024-09-19T04:06:01.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T04:06:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.538 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.539 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.543 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.548 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.548 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.555 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:01 smithi019 ceph-mon[30938]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:06:01.560 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.625 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:01.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:01 smithi111 ceph-mon[35365]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:06:01.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:01 smithi196 ceph-mon[35731]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 -- 172.21.15.19:0/3380737591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8106f10 msgr2=0x7f47c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 --2- 172.21.15.19:0/3380737591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8106f10 0x7f47c810ea90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ca0 tx=0x7f47c002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 -- 172.21.15.19:0/3380737591 shutdown_connections 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 --2- 172.21.15.19:0/3380737591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8106f10 0x7f47c810ea90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 --2- 172.21.15.19:0/3380737591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c8102790 0x7f47c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 --2- 172.21.15.19:0/3380737591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c8101de0 0x7f47c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 -- 172.21.15.19:0/3380737591 >> 172.21.15.19:0/3380737591 conn(0x7f47c80fd8c0 msgr2=0x7f47c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 -- 172.21.15.19:0/3380737591 shutdown_connections 2024-09-19T04:06:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.244+0000 7f47cda39640 1 -- 172.21.15.19:0/3380737591 wait complete. 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47cda39640 1 Processor -- start 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47cda39640 1 -- start start 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47cda39640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c8101de0 0x7f47c8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47cda39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102790 0x7f47c81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47cda39640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c8106f10 0x7f47c819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47cda39640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8111f70 con 0x7f47c8102790 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47cda39640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8111df0 con 0x7f47c8106f10 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47cda39640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81120f0 con 0x7f47c8101de0 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47c67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102790 0x7f47c81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47c67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102790 0x7f47c81988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57444/0 (socket says 172.21.15.19:57444) 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.245+0000 7f47c67fc640 1 -- 172.21.15.19:0/2942773367 learned_addr learned my addr 172.21.15.19:0/2942773367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:02.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c6ffd640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c8101de0 0x7f47c8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c77fe640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c8106f10 0x7f47c819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c67fc640 1 -- 172.21.15.19:0/2942773367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c8101de0 msgr2=0x7f47c8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c67fc640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c8101de0 0x7f47c8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c67fc640 1 -- 172.21.15.19:0/2942773367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c8106f10 msgr2=0x7f47c819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c67fc640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c8106f10 0x7f47c819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c67fc640 1 -- 172.21.15.19:0/2942773367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c819d340 con 0x7f47c8102790 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c77fe640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c8106f10 0x7f47c819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47c67fc640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102790 0x7f47c81988d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c890 tx=0x7f47bc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47cca37640 1 -- 172.21.15.19:0/2942773367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc004220 con 0x7f47c8102790 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c819d630 con 0x7f47c8102790 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47cca37640 1 -- 172.21.15.19:0/2942773367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47bc0043c0 con 0x7f47c8102790 2024-09-19T04:06:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47cca37640 1 -- 172.21.15.19:0/2942773367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc004a00 con 0x7f47c8102790 2024-09-19T04:06:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.246+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c81a4f10 con 0x7f47c8102790 2024-09-19T04:06:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.247+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c81021c0 con 0x7f47c8102790 2024-09-19T04:06:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.248+0000 7f47cca37640 1 -- 172.21.15.19:0/2942773367 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f47bc0079b0 con 0x7f47c8102790 2024-09-19T04:06:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.249+0000 7f47cca37640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f47b00a32d0 0x7f47b00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.249+0000 7f47c6ffd640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f47b00a32d0 0x7f47b00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.249+0000 7f47cca37640 1 -- 172.21.15.19:0/2942773367 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f47bc0cfb20 con 0x7f47c8102790 2024-09-19T04:06:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.250+0000 7f47c6ffd640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f47b00a32d0 0x7f47b00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47b8005fd0 tx=0x7f47b8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.254+0000 7f47cca37640 1 -- 172.21.15.19:0/2942773367 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f47bc014030 con 0x7f47c8102790 2024-09-19T04:06:02.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.434+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f47c8109b60 con 0x7f47c8102790 2024-09-19T04:06:02.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.434+0000 7f47cca37640 1 -- 172.21.15.19:0/2942773367 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f47bc09c3a0 con 0x7f47c8102790 2024-09-19T04:06:02.436 INFO:teuthology.orchestra.run.smithi019.stdout:300647710754 2024-09-19T04:06:02.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f47b00a32d0 msgr2=0x7f47b00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f47b00a32d0 0x7f47b00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47b8005fd0 tx=0x7f47b8005850 comp rx=0 tx=0).stop 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102790 msgr2=0x7f47c81988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102790 0x7f47c81988d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c890 tx=0x7f47bc00cd50 comp rx=0 tx=0).stop 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 shutdown_connections 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f47b00a32d0 0x7f47b00a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c8106f10 0x7f47c819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102790 0x7f47c81988d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 --2- 172.21.15.19:0/2942773367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c8101de0 0x7f47c8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 >> 172.21.15.19:0/2942773367 conn(0x7f47c80fd8c0 msgr2=0x7f47c80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 shutdown_connections 2024-09-19T04:06:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:02.436+0000 7f47cda39640 1 -- 172.21.15.19:0/2942773367 wait complete. 2024-09-19T04:06:02.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:02 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2942773367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:06:02.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:02 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2942773367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:06:02.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:02 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2942773367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T04:06:03.142 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710752 got 300647710754 for osd.10 2024-09-19T04:06:03.142 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.314+0000 7f70edb65640 1 -- 172.21.15.19:0/2179575971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e810c170 msgr2=0x7f70e810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.314+0000 7f70edb65640 1 --2- 172.21.15.19:0/2179575971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e810c170 0x7f70e810e610 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009ca0 tx=0x7f70e002a950 comp rx=0 tx=0).stop 2024-09-19T04:06:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.314+0000 7f70edb65640 1 -- 172.21.15.19:0/2179575971 shutdown_connections 2024-09-19T04:06:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.314+0000 7f70edb65640 1 --2- 172.21.15.19:0/2179575971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e810c170 0x7f70e810e610 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.314+0000 7f70edb65640 1 --2- 172.21.15.19:0/2179575971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e8103660 0x7f70e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.314+0000 7f70edb65640 1 --2- 172.21.15.19:0/2179575971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e8102d40 0x7f70e8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.314+0000 7f70edb65640 1 -- 172.21.15.19:0/2179575971 >> 172.21.15.19:0/2179575971 conn(0x7f70e80fc7a0 msgr2=0x7f70e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.315+0000 7f70edb65640 1 -- 172.21.15.19:0/2179575971 shutdown_connections 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.315+0000 7f70edb65640 1 -- 172.21.15.19:0/2179575971 wait complete. 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.315+0000 7f70edb65640 1 Processor -- start 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.315+0000 7f70edb65640 1 -- start start 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70edb65640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e8102d40 0x7f70e81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70edb65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e8103660 0x7f70e8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70edb65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e810c170 0x7f70e819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70edb65640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8111f70 con 0x7f70e8103660 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70edb65640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8111df0 con 0x7f70e8102d40 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70edb65640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81120f0 con 0x7f70e810c170 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e8103660 0x7f70e8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e8103660 0x7f70e8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57462/0 (socket says 172.21.15.19:57462) 2024-09-19T04:06:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 -- 172.21.15.19:0/1034931566 learned_addr learned my addr 172.21.15.19:0/1034931566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70e77fe640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e8102d40 0x7f70e81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70e7fff640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e810c170 0x7f70e819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 -- 172.21.15.19:0/1034931566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e810c170 msgr2=0x7f70e819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e810c170 0x7f70e819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 -- 172.21.15.19:0/1034931566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e8102d40 msgr2=0x7f70e81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e8102d40 0x7f70e81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 -- 172.21.15.19:0/1034931566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e819d3a0 con 0x7f70e8103660 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70deffd640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e8103660 0x7f70e8198930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c970 tx=0x7f70d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70e57fa640 1 -- 172.21.15.19:0/1034931566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0007bb0 con 0x7f70e8103660 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.316+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e819d690 con 0x7f70e8103660 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.317+0000 7f70e57fa640 1 -- 172.21.15.19:0/1034931566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d0007d50 con 0x7f70e8103660 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.317+0000 7f70e57fa640 1 -- 172.21.15.19:0/1034931566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d000f450 con 0x7f70e8103660 2024-09-19T04:06:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.317+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81a4f70 con 0x7f70e8103660 2024-09-19T04:06:03.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.318+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e8103120 con 0x7f70e8103660 2024-09-19T04:06:03.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.319+0000 7f70e57fa640 1 -- 172.21.15.19:0/1034931566 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f70d0004f40 con 0x7f70e8103660 2024-09-19T04:06:03.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.320+0000 7f70e57fa640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70d40a3350 0x7f70d40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:03.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.320+0000 7f70e57fa640 1 -- 172.21.15.19:0/1034931566 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f70d00cfd10 con 0x7f70e8103660 2024-09-19T04:06:03.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.320+0000 7f70e77fe640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70d40a3350 0x7f70d40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:03.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.320+0000 7f70e77fe640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70d40a3350 0x7f70d40a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009bd0 tx=0x7f70d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:03.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.323+0000 7f70e57fa640 1 -- 172.21.15.19:0/1034931566 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70d0014030 con 0x7f70e8103660 2024-09-19T04:06:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.523+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f70e8199750 con 0x7f70e8103660 2024-09-19T04:06:03.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.524+0000 7f70e57fa640 1 -- 172.21.15.19:0/1034931566 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f70e8199750 con 0x7f70e8103660 2024-09-19T04:06:03.525 INFO:teuthology.orchestra.run.smithi019.stdout:158913790012 2024-09-19T04:06:03.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70d40a3350 msgr2=0x7f70d40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70d40a3350 0x7f70d40a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009bd0 tx=0x7f70d8005f90 comp rx=0 tx=0).stop 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e8103660 msgr2=0x7f70e8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e8103660 0x7f70e8198930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c970 tx=0x7f70d000ce30 comp rx=0 tx=0).stop 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 shutdown_connections 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f70d40a3350 0x7f70d40a5810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e810c170 0x7f70e819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e8103660 0x7f70e8198930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 --2- 172.21.15.19:0/1034931566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e8102d40 0x7f70e81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 >> 172.21.15.19:0/1034931566 conn(0x7f70e80fc7a0 msgr2=0x7f70e80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 shutdown_connections 2024-09-19T04:06:03.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:03.526+0000 7f70edb65640 1 -- 172.21.15.19:0/1034931566 wait complete. 2024-09-19T04:06:03.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:03 smithi019 ceph-mon[30938]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:06:03.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:03 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1034931566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:06:03.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:03 smithi111 ceph-mon[35365]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:06:03.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:03 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1034931566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:06:03.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:03 smithi196 ceph-mon[35731]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-19T04:06:03.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:03 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1034931566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T04:06:04.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.172+0000 7f97257de640 1 -- 172.21.15.19:0/2039747924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720102da0 msgr2=0x7f9720103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:04.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.172+0000 7f97257de640 1 --2- 172.21.15.19:0/2039747924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720102da0 0x7f9720103180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9710009900 tx=0x7f971002a480 comp rx=0 tx=0).stop 2024-09-19T04:06:04.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.173+0000 7f97257de640 1 -- 172.21.15.19:0/2039747924 shutdown_connections 2024-09-19T04:06:04.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.173+0000 7f97257de640 1 --2- 172.21.15.19:0/2039747924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f972010c180 0x7f972010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.173+0000 7f97257de640 1 --2- 172.21.15.19:0/2039747924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97201036c0 0x7f972010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.173+0000 7f97257de640 1 --2- 172.21.15.19:0/2039747924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720102da0 0x7f9720103180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.173+0000 7f97257de640 1 -- 172.21.15.19:0/2039747924 >> 172.21.15.19:0/2039747924 conn(0x7f97200fc7a0 msgr2=0x7f97200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.173+0000 7f97257de640 1 -- 172.21.15.19:0/2039747924 shutdown_connections 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.173+0000 7f97257de640 1 -- 172.21.15.19:0/2039747924 wait complete. 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.173+0000 7f97257de640 1 Processor -- start 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f97257de640 1 -- start start 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f97257de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720102da0 0x7f9720198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f97257de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97201036c0 0x7f97201988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f97257de640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f972010c180 0x7f972019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f97257de640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9720111fb0 con 0x7f97201036c0 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f97257de640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9720111e30 con 0x7f972010c180 2024-09-19T04:06:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f97257de640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9720112130 con 0x7f9720102da0 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97201036c0 0x7f97201988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97201036c0 0x7f97201988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57502/0 (socket says 172.21.15.19:57502) 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971e7fc640 1 -- 172.21.15.19:0/3719861812 learned_addr learned my addr 172.21.15.19:0/3719861812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971effd640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720102da0 0x7f9720198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971f7fe640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f972010c180 0x7f972019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971e7fc640 1 -- 172.21.15.19:0/3719861812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720102da0 msgr2=0x7f9720198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971e7fc640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720102da0 0x7f9720198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971e7fc640 1 -- 172.21.15.19:0/3719861812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f972010c180 msgr2=0x7f972019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971e7fc640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f972010c180 0x7f972019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.174+0000 7f971e7fc640 1 -- 172.21.15.19:0/3719861812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972019d340 con 0x7f97201036c0 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.175+0000 7f971e7fc640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97201036c0 0x7f97201988d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f970800c890 tx=0x7f970800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.175+0000 7f9717fff640 1 -- 172.21.15.19:0/3719861812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708004220 con 0x7f97201036c0 2024-09-19T04:06:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.175+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972019d630 con 0x7f97201036c0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.175+0000 7f9717fff640 1 -- 172.21.15.19:0/3719861812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97080043c0 con 0x7f97201036c0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.175+0000 7f9717fff640 1 -- 172.21.15.19:0/3719861812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708004a00 con 0x7f97201036c0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.175+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97201a4f10 con 0x7f97201036c0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.176+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9720103180 con 0x7f97201036c0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.177+0000 7f9717fff640 1 -- 172.21.15.19:0/3719861812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f97080079b0 con 0x7f97201036c0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.178+0000 7f9717fff640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f970c0a3350 0x7f970c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.178+0000 7f971effd640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f970c0a3350 0x7f970c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.178+0000 7f9717fff640 1 -- 172.21.15.19:0/3719861812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f97080cfab0 con 0x7f97201036c0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.178+0000 7f971effd640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f970c0a3350 0x7f970c0a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9710000c00 tx=0x7f9710005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.182+0000 7f9717fff640 1 -- 172.21.15.19:0/3719861812 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9708014030 con 0x7f97201036c0 2024-09-19T04:06:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.369+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f97201a5230 con 0x7f97201036c0 2024-09-19T04:06:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.369+0000 7f9717fff640 1 -- 172.21.15.19:0/3719861812 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f970809c2d0 con 0x7f97201036c0 2024-09-19T04:06:04.371 INFO:teuthology.orchestra.run.smithi019.stdout:193273528376 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f970c0a3350 msgr2=0x7f970c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f970c0a3350 0x7f970c0a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9710000c00 tx=0x7f9710005df0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97201036c0 msgr2=0x7f97201988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97201036c0 0x7f97201988d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f970800c890 tx=0x7f970800cd50 comp rx=0 tx=0).stop 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 shutdown_connections 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f970c0a3350 0x7f970c0a5810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f972010c180 0x7f972019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97201036c0 0x7f97201988d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 --2- 172.21.15.19:0/3719861812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9720102da0 0x7f9720198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.371+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 >> 172.21.15.19:0/3719861812 conn(0x7f97200fc7a0 msgr2=0x7f97200fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.372+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 shutdown_connections 2024-09-19T04:06:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.372+0000 7f97257de640 1 -- 172.21.15.19:0/3719861812 wait complete. 2024-09-19T04:06:04.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:04 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/598678042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:06:04.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:04 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3719861812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:06:04.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:04 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/598678042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:06:04.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:04 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3719861812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:06:04.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:04 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/598678042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T04:06:04.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:04 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3719861812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T04:06:05.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.204+0000 7f3305a33640 1 -- 172.21.15.19:0/3455398446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300108300 msgr2=0x7f330010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.204+0000 7f3305a33640 1 --2- 172.21.15.19:0/3455398446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300108300 0x7f330010ec70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009930 tx=0x7f32f802a420 comp rx=0 tx=0).stop 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.204+0000 7f3305a33640 1 -- 172.21.15.19:0/3455398446 shutdown_connections 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.204+0000 7f3305a33640 1 --2- 172.21.15.19:0/3455398446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300108300 0x7f330010ec70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.204+0000 7f3305a33640 1 --2- 172.21.15.19:0/3455398446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3300103780 0x7f3300107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.204+0000 7f3305a33640 1 --2- 172.21.15.19:0/3455398446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3300102dd0 0x7f33001031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.204+0000 7f3305a33640 1 -- 172.21.15.19:0/3455398446 >> 172.21.15.19:0/3455398446 conn(0x7f33000fc7d0 msgr2=0x7f33000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.205+0000 7f3305a33640 1 -- 172.21.15.19:0/3455398446 shutdown_connections 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.205+0000 7f3305a33640 1 -- 172.21.15.19:0/3455398446 wait complete. 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.205+0000 7f3305a33640 1 Processor -- start 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.205+0000 7f3305a33640 1 -- start start 2024-09-19T04:06:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f3305a33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3300102dd0 0x7f33001a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f3305a33640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300103780 0x7f33001a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f3305a33640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3300108300 0x7f330019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f3305a33640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33001120d0 con 0x7f3300103780 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f3305a33640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3300111f50 con 0x7f3300108300 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f3305a33640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3300112250 con 0x7f3300102dd0 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300103780 0x7f33001a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300103780 0x7f33001a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57542/0 (socket says 172.21.15.19:57542) 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32fe7fc640 1 -- 172.21.15.19:0/3796680857 learned_addr learned my addr 172.21.15.19:0/3796680857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32feffd640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3300102dd0 0x7f33001a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32ff7fe640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3300108300 0x7f330019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32fe7fc640 1 -- 172.21.15.19:0/3796680857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3300102dd0 msgr2=0x7f33001a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32fe7fc640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3300102dd0 0x7f33001a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32fe7fc640 1 -- 172.21.15.19:0/3796680857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3300108300 msgr2=0x7f330019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32fe7fc640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3300108300 0x7f330019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32fe7fc640 1 -- 172.21.15.19:0/3796680857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330019acc0 con 0x7f3300103780 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.206+0000 7f32ff7fe640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3300108300 0x7f330019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.207+0000 7f32fe7fc640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300103780 0x7f33001a0910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32f400d740 tx=0x7f32f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.207+0000 7f3304a31640 1 -- 172.21.15.19:0/3796680857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4017070 con 0x7f3300103780 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.207+0000 7f3304a31640 1 -- 172.21.15.19:0/3796680857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f400de50 con 0x7f3300103780 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.207+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330019afb0 con 0x7f3300103780 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.207+0000 7f3304a31640 1 -- 172.21.15.19:0/3796680857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f400f6b0 con 0x7f3300103780 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.207+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33001a72a0 con 0x7f3300103780 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.208+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33001031b0 con 0x7f3300103780 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.209+0000 7f3304a31640 1 -- 172.21.15.19:0/3796680857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f32f401a070 con 0x7f3300103780 2024-09-19T04:06:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.210+0000 7f3304a31640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f32e80a3350 0x7f32e80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:05.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.210+0000 7f3304a31640 1 -- 172.21.15.19:0/3796680857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f32f40cf720 con 0x7f3300103780 2024-09-19T04:06:05.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.210+0000 7f32feffd640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f32e80a3350 0x7f32e80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:05.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.210+0000 7f32feffd640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f32e80a3350 0x7f32e80a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f32f0005fd0 tx=0x7f32f0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:05.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.214+0000 7f3304a31640 1 -- 172.21.15.19:0/3796680857 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32f409bfa0 con 0x7f3300103780 2024-09-19T04:06:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 -- 172.21.15.19:0/3752092623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc102e00 msgr2=0x7fd1bc1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 --2- 172.21.15.19:0/3752092623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc102e00 0x7fd1bc1031e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009900 tx=0x7fd1b802a480 comp rx=0 tx=0).stop 2024-09-19T04:06:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 -- 172.21.15.19:0/3752092623 shutdown_connections 2024-09-19T04:06:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 --2- 172.21.15.19:0/3752092623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc10c1a0 0x7fd1bc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 --2- 172.21.15.19:0/3752092623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1bc103720 0x7fd1bc10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 --2- 172.21.15.19:0/3752092623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc102e00 0x7fd1bc1031e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 -- 172.21.15.19:0/3752092623 >> 172.21.15.19:0/3752092623 conn(0x7fd1bc0fc7a0 msgr2=0x7fd1bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 -- 172.21.15.19:0/3752092623 shutdown_connections 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.086+0000 7fd1c4678640 1 -- 172.21.15.19:0/3752092623 wait complete. 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c4678640 1 Processor -- start 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c4678640 1 -- start start 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c4678640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1bc102e00 0x7fd1bc100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c4678640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc103720 0x7fd1bc100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c4678640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc10c1a0 0x7fd1bc1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c4678640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc111f70 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c4678640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc111df0 con 0x7fd1bc103720 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c4678640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1120f0 con 0x7fd1bc102e00 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c2c15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc10c1a0 0x7fd1bc1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c2c15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc10c1a0 0x7fd1bc1014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57478/0 (socket says 172.21.15.19:57478) 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c2c15640 1 -- 172.21.15.19:0/598678042 learned_addr learned my addr 172.21.15.19:0/598678042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c2414640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1bc102e00 0x7fd1bc100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.087+0000 7fd1c1c13640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc103720 0x7fd1bc100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1c2c15640 1 -- 172.21.15.19:0/598678042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1bc102e00 msgr2=0x7fd1bc100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1c2c15640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1bc102e00 0x7fd1bc100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1c2c15640 1 -- 172.21.15.19:0/598678042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc103720 msgr2=0x7fd1bc100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1c2c15640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc103720 0x7fd1bc100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1c2c15640 1 -- 172.21.15.19:0/598678042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc1a68b0 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1c2c15640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc10c1a0 0x7fd1bc1014e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400eaa0 tx=0x7fd1b400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1b37fe640 1 -- 172.21.15.19:0/598678042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4004d20 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1b37fe640 1 -- 172.21.15.19:0/598678042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b4005020 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc1a6ba0 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1b37fe640 1 -- 172.21.15.19:0/598678042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40116d0 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.088+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc1a70e0 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.090+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc1031e0 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.091+0000 7fd1b37fe640 1 -- 172.21.15.19:0/598678042 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd1b40051d0 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.092+0000 7fd1b37fe640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd1a80a3350 0x7fd1a80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.092+0000 7fd1c2414640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd1a80a3350 0x7fd1a80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.092+0000 7fd1b37fe640 1 -- 172.21.15.19:0/598678042 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fd1b40cfef0 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.093+0000 7fd1c2414640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd1a80a3350 0x7fd1a80a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009230 tx=0x7fd1b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.096+0000 7fd1b37fe640 1 -- 172.21.15.19:0/598678042 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd1b4016030 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.277+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd1bc107280 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stdout:137438953538 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.277+0000 7fd1b37fe640 1 -- 172.21.15.19:0/598678042 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd1b409c710 con 0x7fd1bc10c1a0 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd1a80a3350 msgr2=0x7fd1a80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd1a80a3350 0x7fd1a80a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009230 tx=0x7fd1b80058d0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc10c1a0 msgr2=0x7fd1bc1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc10c1a0 0x7fd1bc1014e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400eaa0 tx=0x7fd1b400eea0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 shutdown_connections 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd1a80a3350 0x7fd1a80a5810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1bc10c1a0 0x7fd1bc1014e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1bc103720 0x7fd1bc100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 --2- 172.21.15.19:0/598678042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1bc102e00 0x7fd1bc100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 >> 172.21.15.19:0/598678042 conn(0x7fd1bc0fc7a0 msgr2=0x7fd1bc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 shutdown_connections 2024-09-19T04:06:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.280+0000 7fd1c4678640 1 -- 172.21.15.19:0/598678042 wait complete. 2024-09-19T04:06:05.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.392+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f330010c2a0 con 0x7f3300103780 2024-09-19T04:06:05.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.392+0000 7f3304a31640 1 -- 172.21.15.19:0/3796680857 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f32f409b8a0 con 0x7f3300103780 2024-09-19T04:06:05.394 INFO:teuthology.orchestra.run.smithi019.stdout:107374182471 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f32e80a3350 msgr2=0x7f32e80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f32e80a3350 0x7f32e80a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f32f0005fd0 tx=0x7f32f0009680 comp rx=0 tx=0).stop 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300103780 msgr2=0x7f33001a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300103780 0x7f33001a0910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32f400d740 tx=0x7f32f400dc00 comp rx=0 tx=0).stop 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 shutdown_connections 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f32e80a3350 0x7f32e80a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3300108300 0x7f330019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3300103780 0x7f33001a0910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 --2- 172.21.15.19:0/3796680857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3300102dd0 0x7f33001a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.394+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 >> 172.21.15.19:0/3796680857 conn(0x7f33000fc7d0 msgr2=0x7f3300101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:05.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.395+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 shutdown_connections 2024-09-19T04:06:05.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:05.395+0000 7f3305a33640 1 -- 172.21.15.19:0/3796680857 wait complete. 2024-09-19T04:06:05.835 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:05 smithi019 ceph-mon[30938]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:06:05.835 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:05 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4282071898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:06:05.835 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:05 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3796680857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:06:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:05 smithi111 ceph-mon[35365]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:06:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:05 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/4282071898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:06:05.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:05 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3796680857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:06:05.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:05 smithi196 ceph-mon[35731]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T04:06:05.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:05 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/4282071898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T04:06:05.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:05 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3796680857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T04:06:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.401+0000 7f9f9f8db640 1 -- 172.21.15.19:0/817512120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98106f00 msgr2=0x7f9f9810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.401+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/817512120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98106f00 0x7f9f9810ea80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009ca0 tx=0x7f9f9002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 -- 172.21.15.19:0/817512120 shutdown_connections 2024-09-19T04:06:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/817512120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98106f00 0x7f9f9810ea80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/817512120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98102780 0x7f9f981067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/817512120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f98101dd0 0x7f9f981021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 -- 172.21.15.19:0/817512120 >> 172.21.15.19:0/817512120 conn(0x7f9f980fd8b0 msgr2=0x7f9f980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 -- 172.21.15.19:0/817512120 shutdown_connections 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 -- 172.21.15.19:0/817512120 wait complete. 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 Processor -- start 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.402+0000 7f9f9f8db640 1 -- start start 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9f8db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101dd0 0x7f9f9819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9f8db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f98102780 0x7f9f9819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9f8db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98106f00 0x7f9f9819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9f8db640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98074ce0 con 0x7f9f98101dd0 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9f8db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98074b60 con 0x7f9f98102780 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9f8db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98074e60 con 0x7f9f98106f00 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9d677640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101dd0 0x7f9f9819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9d677640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101dd0 0x7f9f9819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57518/0 (socket says 172.21.15.19:57518) 2024-09-19T04:06:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9d677640 1 -- 172.21.15.19:0/4282071898 learned_addr learned my addr 172.21.15.19:0/4282071898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9ce76640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f98102780 0x7f9f9819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9de78640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98106f00 0x7f9f9819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:06.350 INFO:teuthology.orchestra.run.smithi019.stdout:279172874281 2024-09-19T04:06:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9d677640 1 -- 172.21.15.19:0/4282071898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98106f00 msgr2=0x7f9f9819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9d677640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98106f00 0x7f9f9819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9d677640 1 -- 172.21.15.19:0/4282071898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f98102780 msgr2=0x7f9f9819aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9d677640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f98102780 0x7f9f9819aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9d677640 1 -- 172.21.15.19:0/4282071898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9819f580 con 0x7f9f98101dd0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.403+0000 7f9f9ce76640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f98102780 0x7f9f9819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.404+0000 7f9f9d677640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101dd0 0x7f9f9819a990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400e9f0 tx=0x7f9f9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.404+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/4282071898 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400cd80 con 0x7f9f98101dd0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.404+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9819f870 con 0x7f9f98101dd0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.404+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/4282071898 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f94010910 con 0x7f9f98101dd0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.404+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/4282071898 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f940188f0 con 0x7f9f98101dd0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.404+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a7150 con 0x7f9f98101dd0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.405+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f981021b0 con 0x7f9f98101dd0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.406+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/4282071898 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9f94010ac0 con 0x7f9f98101dd0 2024-09-19T04:06:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.407+0000 7f9f8e7fc640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9f840a32d0 0x7f9f840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.407+0000 7f9f9ce76640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9f840a32d0 0x7f9f840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.407+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/4282071898 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f9f940d0570 con 0x7f9f98101dd0 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.408+0000 7f9f9ce76640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9f840a32d0 0x7f9f840a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f98075f90 tx=0x7f9f88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.411+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/4282071898 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f9409c7a0 con 0x7f9f98101dd0 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.594+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f9f981a73f0 con 0x7f9f98101dd0 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.594+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/4282071898 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9f940a0aa0 con 0x7f9f98101dd0 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9f840a32d0 msgr2=0x7f9f840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9f840a32d0 0x7f9f840a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f98075f90 tx=0x7f9f88005f90 comp rx=0 tx=0).stop 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101dd0 msgr2=0x7f9f9819a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101dd0 0x7f9f9819a990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400e9f0 tx=0x7f9f9400eeb0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 shutdown_connections 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9f840a32d0 0x7f9f840a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98106f00 0x7f9f9819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f98102780 0x7f9f9819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 --2- 172.21.15.19:0/4282071898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101dd0 0x7f9f9819a990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 >> 172.21.15.19:0/4282071898 conn(0x7f9f980fd8b0 msgr2=0x7f9f981057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 shutdown_connections 2024-09-19T04:06:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:04.595+0000 7f9f9f8db640 1 -- 172.21.15.19:0/4282071898 wait complete. 2024-09-19T04:06:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 -- 172.21.15.19:0/1218645010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 msgr2=0x7f907010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 --2- 172.21.15.19:0/1218645010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 0x7f907010d950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9068009cd0 tx=0x7f906802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 -- 172.21.15.19:0/1218645010 shutdown_connections 2024-09-19T04:06:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 --2- 172.21.15.19:0/1218645010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 0x7f907010d950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 --2- 172.21.15.19:0/1218645010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9070100a80 0x7f9070100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 --2- 172.21.15.19:0/1218645010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9070106a80 0x7f9070106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 -- 172.21.15.19:0/1218645010 >> 172.21.15.19:0/1218645010 conn(0x7f90700fc7a0 msgr2=0x7f90700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 -- 172.21.15.19:0/1218645010 shutdown_connections 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.546+0000 7f9075529640 1 -- 172.21.15.19:0/1218645010 wait complete. 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f9075529640 1 Processor -- start 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f9075529640 1 -- start start 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f9075529640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9070100a80 0x7f9070197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f9075529640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 0x7f90701975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f9075529640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9070106a80 0x7f9070197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f9075529640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9070110ed0 con 0x7f9070101420 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f9075529640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9070110d50 con 0x7f9070106a80 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f9075529640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9070111050 con 0x7f9070100a80 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f906e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 0x7f90701975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f906e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 0x7f90701975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57568/0 (socket says 172.21.15.19:57568) 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f906e7fc640 1 -- 172.21.15.19:0/2385340979 learned_addr learned my addr 172.21.15.19:0/2385340979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.547+0000 7f906effd640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9070100a80 0x7f9070197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f906f7fe640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9070106a80 0x7f9070197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f906e7fc640 1 -- 172.21.15.19:0/2385340979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9070100a80 msgr2=0x7f9070197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f906e7fc640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9070100a80 0x7f9070197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f906e7fc640 1 -- 172.21.15.19:0/2385340979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9070106a80 msgr2=0x7f9070197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f906e7fc640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9070106a80 0x7f9070197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f906e7fc640 1 -- 172.21.15.19:0/2385340979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90701a68e0 con 0x7f9070101420 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f906e7fc640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 0x7f90701975b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f906400d770 tx=0x7f906400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f9057fff640 1 -- 172.21.15.19:0/2385340979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064017070 con 0x7f9070101420 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f9057fff640 1 -- 172.21.15.19:0/2385340979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064005500 con 0x7f9070101420 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90701a6bd0 con 0x7f9070101420 2024-09-19T04:06:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f9057fff640 1 -- 172.21.15.19:0/2385340979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064004e00 con 0x7f9070101420 2024-09-19T04:06:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.548+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a70c0 con 0x7f9070101420 2024-09-19T04:06:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.549+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9070100ee0 con 0x7f9070101420 2024-09-19T04:06:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.551+0000 7f9057fff640 1 -- 172.21.15.19:0/2385340979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9064004fa0 con 0x7f9070101420 2024-09-19T04:06:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.552+0000 7f9057fff640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90580a32d0 0x7f90580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.552+0000 7f906effd640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90580a32d0 0x7f90580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.552+0000 7f9057fff640 1 -- 172.21.15.19:0/2385340979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f90640cfc70 con 0x7f9070101420 2024-09-19T04:06:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.552+0000 7f906effd640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90580a32d0 0x7f90580a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f90600058c0 tx=0x7f906000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.555+0000 7f9057fff640 1 -- 172.21.15.19:0/2385340979 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f906409c490 con 0x7f9070101420 2024-09-19T04:06:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.732+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f90701a7430 con 0x7f9070101420 2024-09-19T04:06:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.732+0000 7f9057fff640 1 -- 172.21.15.19:0/2385340979 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f906409bd90 con 0x7f9070101420 2024-09-19T04:06:06.734 INFO:teuthology.orchestra.run.smithi019.stdout:214748364851 2024-09-19T04:06:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90580a32d0 msgr2=0x7f90580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90580a32d0 0x7f90580a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f90600058c0 tx=0x7f906000a2d0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 msgr2=0x7f90701975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 0x7f90701975b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f906400d770 tx=0x7f906400dc30 comp rx=0 tx=0).stop 2024-09-19T04:06:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 shutdown_connections 2024-09-19T04:06:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90580a32d0 0x7f90580a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9070106a80 0x7f9070197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9070101420 0x7f90701975b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 --2- 172.21.15.19:0/2385340979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9070100a80 0x7f9070197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 >> 172.21.15.19:0/2385340979 conn(0x7f90700fc7a0 msgr2=0x7f90700fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 shutdown_connections 2024-09-19T04:06:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:06.735+0000 7f9075529640 1 -- 172.21.15.19:0/2385340979 wait complete. 2024-09-19T04:06:07.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874279 got 279172874281 for osd.9 2024-09-19T04:06:07.279 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:07.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:07 smithi019 ceph-mon[30938]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-19T04:06:07.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:07 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2385340979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:06:07.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182469 got 107374182471 for osd.3 2024-09-19T04:06:07.707 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:07.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:07 smithi111 ceph-mon[35365]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-19T04:06:07.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:07 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2385340979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:06:07.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:07 smithi196 ceph-mon[35731]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-19T04:06:07.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:07 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2385340979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T04:06:08.530 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364847 got 214748364851 for osd.7 2024-09-19T04:06:08.530 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:09.514 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528374 got 193273528376 for osd.6 2024-09-19T04:06:09.514 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:09.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:09 smithi019 ceph-mon[30938]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T04:06:09.887 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:09 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1971286949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:06:09.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:09 smithi111 ceph-mon[35365]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T04:06:09.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:09 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1971286949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:06:09.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790011 got 158913790012 for osd.5 2024-09-19T04:06:09.915 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:09.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:09 smithi196 ceph-mon[35731]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T04:06:09.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:09 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1971286949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T04:06:10.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 -- 172.21.15.19:0/966901426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c101440 msgr2=0x7f352c10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 --2- 172.21.15.19:0/966901426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c101440 0x7f352c10df30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3520009ca0 tx=0x7f352002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 -- 172.21.15.19:0/966901426 shutdown_connections 2024-09-19T04:06:10.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 --2- 172.21.15.19:0/966901426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c10e500 0x7f352c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 --2- 172.21.15.19:0/966901426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c101440 0x7f352c10df30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 --2- 172.21.15.19:0/966901426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f352c100b20 0x7f352c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 -- 172.21.15.19:0/966901426 >> 172.21.15.19:0/966901426 conn(0x7f352c0fc7d0 msgr2=0x7f352c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 -- 172.21.15.19:0/966901426 shutdown_connections 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 -- 172.21.15.19:0/966901426 wait complete. 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.081+0000 7f353193b640 1 Processor -- start 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f353193b640 1 -- start start 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f353193b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f352c100b20 0x7f352c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f353193b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c101440 0x7f352c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f353193b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c10e500 0x7f352c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f353193b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f352c114250 con 0x7f352c10e500 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f353193b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1140d0 con 0x7f352c100b20 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f353193b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1143d0 con 0x7f352c101440 2024-09-19T04:06:10.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c10e500 0x7f352c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c10e500 0x7f352c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57588/0 (socket says 172.21.15.19:57588) 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 -- 172.21.15.19:0/3652220916 learned_addr learned my addr 172.21.15.19:0/3652220916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352affd640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f352c100b20 0x7f352c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352a7fc640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c101440 0x7f352c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 -- 172.21.15.19:0/3652220916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c101440 msgr2=0x7f352c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c101440 0x7f352c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 -- 172.21.15.19:0/3652220916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f352c100b20 msgr2=0x7f352c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f352c100b20 0x7f352c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 -- 172.21.15.19:0/3652220916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c19f650 con 0x7f352c10e500 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.082+0000 7f352b7fe640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c10e500 0x7f352c19ef70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f352400c970 tx=0x7f352400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.083+0000 7f3530939640 1 -- 172.21.15.19:0/3652220916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524007bb0 con 0x7f352c10e500 2024-09-19T04:06:10.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.083+0000 7f3530939640 1 -- 172.21.15.19:0/3652220916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524007d50 con 0x7f352c10e500 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.083+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c19f940 con 0x7f352c10e500 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.083+0000 7f3530939640 1 -- 172.21.15.19:0/3652220916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524018750 con 0x7f352c10e500 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.083+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c1a7220 con 0x7f352c10e500 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.084+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352c100f00 con 0x7f352c10e500 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.085+0000 7f3530939640 1 -- 172.21.15.19:0/3652220916 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f352400f450 con 0x7f352c10e500 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.086+0000 7f3530939640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f35140a3350 0x7f35140a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.086+0000 7f352affd640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f35140a3350 0x7f35140a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.086+0000 7f3530939640 1 -- 172.21.15.19:0/3652220916 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f35240cfe40 con 0x7f352c10e500 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.086+0000 7f352affd640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f35140a3350 0x7f35140a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f351c00abe0 tx=0x7f351c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.090+0000 7f3530939640 1 -- 172.21.15.19:0/3652220916 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3524014030 con 0x7f352c10e500 2024-09-19T04:06:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.267+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f352c109600 con 0x7f352c10e500 2024-09-19T04:06:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.268+0000 7f3530939640 1 -- 172.21.15.19:0/3652220916 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f352409c6c0 con 0x7f352c10e500 2024-09-19T04:06:10.269 INFO:teuthology.orchestra.run.smithi019.stdout:55834574930 2024-09-19T04:06:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f35140a3350 msgr2=0x7f35140a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f35140a3350 0x7f35140a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f351c00abe0 tx=0x7f351c00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c10e500 msgr2=0x7f352c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c10e500 0x7f352c19ef70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f352400c970 tx=0x7f352400cd70 comp rx=0 tx=0).stop 2024-09-19T04:06:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 shutdown_connections 2024-09-19T04:06:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f35140a3350 0x7f35140a5810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f352c10e500 0x7f352c19ef70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c101440 0x7f352c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 --2- 172.21.15.19:0/3652220916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f352c100b20 0x7f352c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 >> 172.21.15.19:0/3652220916 conn(0x7f352c0fc7d0 msgr2=0x7f352c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 shutdown_connections 2024-09-19T04:06:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.270+0000 7f353193b640 1 -- 172.21.15.19:0/3652220916 wait complete. 2024-09-19T04:06:10.602 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953535 got 137438953538 for osd.4 2024-09-19T04:06:10.602 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 -- 172.21.15.19:0/1261073894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58106b80 msgr2=0x7efc58106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/1261073894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58106b80 0x7efc58106f60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efc48009900 tx=0x7efc4802a410 comp rx=0 tx=0).stop 2024-09-19T04:06:10.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 -- 172.21.15.19:0/1261073894 shutdown_connections 2024-09-19T04:06:10.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/1261073894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc58100500 0x7efc5810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/1261073894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc580ffb60 0x7efc580fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/1261073894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58106b80 0x7efc58106f60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 -- 172.21.15.19:0/1261073894 >> 172.21.15.19:0/1261073894 conn(0x7efc580fba90 msgr2=0x7efc580fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:10.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 -- 172.21.15.19:0/1261073894 shutdown_connections 2024-09-19T04:06:10.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.757+0000 7efc5f7e7640 1 -- 172.21.15.19:0/1261073894 wait complete. 2024-09-19T04:06:10.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5f7e7640 1 Processor -- start 2024-09-19T04:06:10.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5f7e7640 1 -- start start 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5f7e7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc580ffb60 0x7efc58198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5f7e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58100500 0x7efc58198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5f7e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58106b80 0x7efc5819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5f7e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc58111f60 con 0x7efc58100500 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5f7e7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc58111de0 con 0x7efc580ffb60 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5f7e7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc581120e0 con 0x7efc58106b80 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5cd82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58100500 0x7efc58198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5cd82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58100500 0x7efc58198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57614/0 (socket says 172.21.15.19:57614) 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.758+0000 7efc5cd82640 1 -- 172.21.15.19:0/472973405 learned_addr learned my addr 172.21.15.19:0/472973405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5d583640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc580ffb60 0x7efc58198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5dd84640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58106b80 0x7efc5819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5cd82640 1 -- 172.21.15.19:0/472973405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58106b80 msgr2=0x7efc5819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5cd82640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58106b80 0x7efc5819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5cd82640 1 -- 172.21.15.19:0/472973405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc580ffb60 msgr2=0x7efc58198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5cd82640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc580ffb60 0x7efc58198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5cd82640 1 -- 172.21.15.19:0/472973405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5819d3b0 con 0x7efc58100500 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5dd84640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58106b80 0x7efc5819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5d583640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc580ffb60 0x7efc58198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5cd82640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58100500 0x7efc58198940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efc5400d950 tx=0x7efc5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc4e7fc640 1 -- 172.21.15.19:0/472973405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54017030 con 0x7efc58100500 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc4e7fc640 1 -- 172.21.15.19:0/472973405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc54004020 con 0x7efc58100500 2024-09-19T04:06:10.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5819d6a0 con 0x7efc58100500 2024-09-19T04:06:10.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc4e7fc640 1 -- 172.21.15.19:0/472973405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc540047a0 con 0x7efc58100500 2024-09-19T04:06:10.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.759+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc581a4f80 con 0x7efc58100500 2024-09-19T04:06:10.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.760+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc580fffc0 con 0x7efc58100500 2024-09-19T04:06:10.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.762+0000 7efc4e7fc640 1 -- 172.21.15.19:0/472973405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efc540041d0 con 0x7efc58100500 2024-09-19T04:06:10.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.762+0000 7efc4e7fc640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efc440a32d0 0x7efc440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.763+0000 7efc5d583640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efc440a32d0 0x7efc440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.763+0000 7efc4e7fc640 1 -- 172.21.15.19:0/472973405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7efc540cf900 con 0x7efc58100500 2024-09-19T04:06:10.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.763+0000 7efc5d583640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efc440a32d0 0x7efc440a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc48000c00 tx=0x7efc48004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.767+0000 7efc4e7fc640 1 -- 172.21.15.19:0/472973405 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc5409c180 con 0x7efc58100500 2024-09-19T04:06:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 -- 172.21.15.19:0/2089941901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 msgr2=0x7fcce810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 --2- 172.21.15.19:0/2089941901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 0x7fcce810df30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009900 tx=0x7fccd802a410 comp rx=0 tx=0).stop 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 -- 172.21.15.19:0/2089941901 shutdown_connections 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 --2- 172.21.15.19:0/2089941901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcce810e500 0x7fcce8110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 --2- 172.21.15.19:0/2089941901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 0x7fcce810df30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 --2- 172.21.15.19:0/2089941901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce8100b20 0x7fcce8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 -- 172.21.15.19:0/2089941901 >> 172.21.15.19:0/2089941901 conn(0x7fcce80fc7d0 msgr2=0x7fcce80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 -- 172.21.15.19:0/2089941901 shutdown_connections 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.004+0000 7fccef943640 1 -- 172.21.15.19:0/2089941901 wait complete. 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.005+0000 7fccef943640 1 Processor -- start 2024-09-19T04:06:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.005+0000 7fccef943640 1 -- start start 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccef943640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcce8100b20 0x7fcce81a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccef943640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 0x7fcce81a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccef943640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce810e500 0x7fcce819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccef943640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8114250 con 0x7fcce8101440 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccef943640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81140d0 con 0x7fcce810e500 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccef943640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81143d0 con 0x7fcce8100b20 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccecede640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 0x7fcce81a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccecede640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 0x7fcce81a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57576/0 (socket says 172.21.15.19:57576) 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccecede640 1 -- 172.21.15.19:0/1971286949 learned_addr learned my addr 172.21.15.19:0/1971286949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccedee0640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce810e500 0x7fcce819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fcced6df640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcce8100b20 0x7fcce81a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccecede640 1 -- 172.21.15.19:0/1971286949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcce8100b20 msgr2=0x7fcce81a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccecede640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcce8100b20 0x7fcce81a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccecede640 1 -- 172.21.15.19:0/1971286949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce810e500 msgr2=0x7fcce819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccecede640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce810e500 0x7fcce819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.006+0000 7fccecede640 1 -- 172.21.15.19:0/1971286949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce819acf0 con 0x7fcce8101440 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.007+0000 7fccecede640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 0x7fcce81a0940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005940 tx=0x7fccd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.007+0000 7fccedee0640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce810e500 0x7fcce819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.007+0000 7fccde7fc640 1 -- 172.21.15.19:0/1971286949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd803f030 con 0x7fcce8101440 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.007+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce819af80 con 0x7fcce8101440 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.007+0000 7fccde7fc640 1 -- 172.21.15.19:0/1971286949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd8003ad0 con 0x7fcce8101440 2024-09-19T04:06:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.007+0000 7fccde7fc640 1 -- 172.21.15.19:0/1971286949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8035070 con 0x7fcce8101440 2024-09-19T04:06:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.007+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce81a72d0 con 0x7fcce8101440 2024-09-19T04:06:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.008+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce8100f00 con 0x7fcce8101440 2024-09-19T04:06:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.009+0000 7fccde7fc640 1 -- 172.21.15.19:0/1971286949 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fccd8042030 con 0x7fcce8101440 2024-09-19T04:06:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.010+0000 7fccde7fc640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fccd40a3350 0x7fccd40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.010+0000 7fcced6df640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fccd40a3350 0x7fccd40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.011+0000 7fccde7fc640 1 -- 172.21.15.19:0/1971286949 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fccd80f0660 con 0x7fcce8101440 2024-09-19T04:06:10.830 INFO:teuthology.orchestra.run.smithi019.stdout:34359738455 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.011+0000 7fcced6df640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fccd40a3350 0x7fccd40a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009bd0 tx=0x7fcce4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.014+0000 7fccde7fc640 1 -- 172.21.15.19:0/1971286949 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fccd80bc830 con 0x7fcce8101440 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.208+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fcce8109600 con 0x7fcce8101440 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.209+0000 7fccde7fc640 1 -- 172.21.15.19:0/1971286949 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fccd80c0b30 con 0x7fcce8101440 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fccd40a3350 msgr2=0x7fccd40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fccd40a3350 0x7fccd40a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009bd0 tx=0x7fcce4005f90 comp rx=0 tx=0).stop 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 msgr2=0x7fcce81a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 0x7fcce81a0940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005940 tx=0x7fccd8004270 comp rx=0 tx=0).stop 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 shutdown_connections 2024-09-19T04:06:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fccd40a3350 0x7fccd40a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce810e500 0x7fcce819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce8101440 0x7fcce81a0940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 --2- 172.21.15.19:0/1971286949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcce8100b20 0x7fcce81a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 >> 172.21.15.19:0/1971286949 conn(0x7fcce80fc7d0 msgr2=0x7fcce80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 shutdown_connections 2024-09-19T04:06:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:09.211+0000 7fccef943640 1 -- 172.21.15.19:0/1971286949 wait complete. 2024-09-19T04:06:10.832 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:10 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3652220916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:06:10.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:10 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3652220916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:06:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.945+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7efc5802d7c0 con 0x7efc58100500 2024-09-19T04:06:10.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.945+0000 7efc4e7fc640 1 -- 172.21.15.19:0/472973405 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc5409ba80 con 0x7efc58100500 2024-09-19T04:06:10.947 INFO:teuthology.orchestra.run.smithi019.stdout:77309411405 2024-09-19T04:06:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efc440a32d0 msgr2=0x7efc440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efc440a32d0 0x7efc440a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc48000c00 tx=0x7efc48004100 comp rx=0 tx=0).stop 2024-09-19T04:06:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58100500 msgr2=0x7efc58198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58100500 0x7efc58198940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efc5400d950 tx=0x7efc5400de10 comp rx=0 tx=0).stop 2024-09-19T04:06:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 shutdown_connections 2024-09-19T04:06:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efc440a32d0 0x7efc440a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58106b80 0x7efc5819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58100500 0x7efc58198940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 --2- 172.21.15.19:0/472973405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc580ffb60 0x7efc58198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 >> 172.21.15.19:0/472973405 conn(0x7efc580fba90 msgr2=0x7efc580fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 shutdown_connections 2024-09-19T04:06:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:10.948+0000 7efc5f7e7640 1 -- 172.21.15.19:0/472973405 wait complete. 2024-09-19T04:06:10.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:10 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3652220916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T04:06:11.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 -- 172.21.15.19:0/2625988303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 msgr2=0x7fa14810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/2625988303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 0x7fa14810bb50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009900 tx=0x7fa13c02a480 comp rx=0 tx=0).stop 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 -- 172.21.15.19:0/2625988303 shutdown_connections 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/2625988303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa14810c170 0x7fa14810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/2625988303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 0x7fa14810bb50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/2625988303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa148102d40 0x7fa148103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 -- 172.21.15.19:0/2625988303 >> 172.21.15.19:0/2625988303 conn(0x7fa1480fc7a0 msgr2=0x7fa1480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 -- 172.21.15.19:0/2625988303 shutdown_connections 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 -- 172.21.15.19:0/2625988303 wait complete. 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 Processor -- start 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.590+0000 7fa14e7ec640 1 -- start start 2024-09-19T04:06:11.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa14e7ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148102d40 0x7fa1481983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa14e7ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 0x7fa148198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa14e7ec640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14810c170 0x7fa14819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa14e7ec640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa148111f70 con 0x7fa148103660 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa14e7ec640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa148111df0 con 0x7fa14810c170 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa14e7ec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481120f0 con 0x7fa148102d40 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 0x7fa148198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 0x7fa148198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38398/0 (socket says 172.21.15.19:38398) 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 -- 172.21.15.19:0/3225279016 learned_addr learned my addr 172.21.15.19:0/3225279016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa147fff640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148102d40 0x7fa1481983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa14cd89640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14810c170 0x7fa14819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 -- 172.21.15.19:0/3225279016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148102d40 msgr2=0x7fa1481983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148102d40 0x7fa1481983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 -- 172.21.15.19:0/3225279016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14810c170 msgr2=0x7fa14819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14810c170 0x7fa14819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 -- 172.21.15.19:0/3225279016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14819d3a0 con 0x7fa148103660 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.591+0000 7fa1477fe640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 0x7fa148198930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa13c006010 tx=0x7fa13c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.592+0000 7fa1457fa640 1 -- 172.21.15.19:0/3225279016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c03f030 con 0x7fa148103660 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.592+0000 7fa1457fa640 1 -- 172.21.15.19:0/3225279016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa13c003ce0 con 0x7fa148103660 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.592+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14819d630 con 0x7fa148103660 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.592+0000 7fa1457fa640 1 -- 172.21.15.19:0/3225279016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c035070 con 0x7fa148103660 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.592+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a4f70 con 0x7fa148103660 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.593+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148103120 con 0x7fa148103660 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.594+0000 7fa1457fa640 1 -- 172.21.15.19:0/3225279016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa13c042030 con 0x7fa148103660 2024-09-19T04:06:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.594+0000 7fa1457fa640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa1300a32d0 0x7fa1300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.595+0000 7fa1457fa640 1 -- 172.21.15.19:0/3225279016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fa13c0f0310 con 0x7fa148103660 2024-09-19T04:06:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.595+0000 7fa147fff640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa1300a32d0 0x7fa1300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.595+0000 7fa147fff640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa1300a32d0 0x7fa1300a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa138005d00 tx=0x7fa13800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.599+0000 7fa1457fa640 1 -- 172.21.15.19:0/3225279016 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa13c0bc600 con 0x7fa148103660 2024-09-19T04:06:11.607 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:11.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:11 smithi019 ceph-mon[30938]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:11.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:11 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/472973405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:06:11.786 INFO:teuthology.orchestra.run.smithi019.stdout:330712481824 2024-09-19T04:06:11.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.784+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fa1481071e0 con 0x7fa148103660 2024-09-19T04:06:11.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.785+0000 7fa1457fa640 1 -- 172.21.15.19:0/3225279016 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa13c0c0900 con 0x7fa148103660 2024-09-19T04:06:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa1300a32d0 msgr2=0x7fa1300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa1300a32d0 0x7fa1300a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa138005d00 tx=0x7fa13800b410 comp rx=0 tx=0).stop 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 msgr2=0x7fa148198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 0x7fa148198930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa13c006010 tx=0x7fa13c0042f0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 shutdown_connections 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa1300a32d0 0x7fa1300a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14810c170 0x7fa14819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa148103660 0x7fa148198930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 --2- 172.21.15.19:0/3225279016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148102d40 0x7fa1481983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 >> 172.21.15.19:0/3225279016 conn(0x7fa1480fc7a0 msgr2=0x7fa1480fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 shutdown_connections 2024-09-19T04:06:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:11.787+0000 7fa14e7ec640 1 -- 172.21.15.19:0/3225279016 wait complete. 2024-09-19T04:06:11.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:11 smithi111 ceph-mon[35365]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:11.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:11 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/472973405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:06:11.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:11 smithi196 ceph-mon[35731]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:11.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:11 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/472973405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T04:06:12.142 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738454 got 34359738455 for osd.0 2024-09-19T04:06:12.143 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:12.795 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411405 for osd.2 2024-09-19T04:06:12.795 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:12.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:12 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3225279016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:06:12.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:12 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3225279016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:06:12.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:12 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3225279016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T04:06:13.467 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574930 for osd.1 2024-09-19T04:06:13.467 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:13.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:13 smithi019 ceph-mon[30938]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:13.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:13 smithi111 ceph-mon[35365]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:13.926 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481820 got 330712481824 for osd.11 2024-09-19T04:06:13.927 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:13.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:13 smithi196 ceph-mon[35731]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.548+0000 7fb38c86c640 1 -- 172.21.15.19:0/1462537271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881036e0 msgr2=0x7fb388107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.548+0000 7fb38c86c640 1 --2- 172.21.15.19:0/1462537271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881036e0 0x7fb388107b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb370009930 tx=0x7fb37002a420 comp rx=0 tx=0).stop 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.549+0000 7fb38c86c640 1 -- 172.21.15.19:0/1462537271 shutdown_connections 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.549+0000 7fb38c86c640 1 --2- 172.21.15.19:0/1462537271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3881082a0 0x7fb38810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.549+0000 7fb38c86c640 1 --2- 172.21.15.19:0/1462537271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881036e0 0x7fb388107b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.549+0000 7fb38c86c640 1 --2- 172.21.15.19:0/1462537271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb388102d30 0x7fb388103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.549+0000 7fb38c86c640 1 -- 172.21.15.19:0/1462537271 >> 172.21.15.19:0/1462537271 conn(0x7fb3880fc7d0 msgr2=0x7fb3880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.549+0000 7fb38c86c640 1 -- 172.21.15.19:0/1462537271 shutdown_connections 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.549+0000 7fb38c86c640 1 -- 172.21.15.19:0/1462537271 wait complete. 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38c86c640 1 Processor -- start 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38c86c640 1 -- start start 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38c86c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb388102d30 0x7fb3881a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38c86c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3881036e0 0x7fb3881a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38c86c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881082a0 0x7fb38819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38c86c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881120a0 con 0x7fb3881082a0 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38c86c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb388111f20 con 0x7fb388102d30 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38c86c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb388112220 con 0x7fb3881036e0 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb386d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881082a0 0x7fb38819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb386d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881082a0 0x7fb38819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38404/0 (socket says 172.21.15.19:38404) 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb386d9d640 1 -- 172.21.15.19:0/2834992817 learned_addr learned my addr 172.21.15.19:0/2834992817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb37ffff640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3881036e0 0x7fb3881a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb38659c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb388102d30 0x7fb3881a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb386d9d640 1 -- 172.21.15.19:0/2834992817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3881036e0 msgr2=0x7fb3881a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb386d9d640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3881036e0 0x7fb3881a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb386d9d640 1 -- 172.21.15.19:0/2834992817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb388102d30 msgr2=0x7fb3881a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb386d9d640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb388102d30 0x7fb3881a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.550+0000 7fb386d9d640 1 -- 172.21.15.19:0/2834992817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38819ad00 con 0x7fb3881082a0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.551+0000 7fb386d9d640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881082a0 0x7fb38819a5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb380009fd0 tx=0x7fb38000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.551+0000 7fb37f7fe640 1 -- 172.21.15.19:0/2834992817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380017070 con 0x7fb3881082a0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.551+0000 7fb37f7fe640 1 -- 172.21.15.19:0/2834992817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb380003e50 con 0x7fb3881082a0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.551+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38819aff0 con 0x7fb3881082a0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.551+0000 7fb37f7fe640 1 -- 172.21.15.19:0/2834992817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3800106c0 con 0x7fb3881082a0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.551+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3881a72e0 con 0x7fb3881082a0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.552+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb388103110 con 0x7fb3881082a0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.553+0000 7fb37f7fe640 1 -- 172.21.15.19:0/2834992817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb380003970 con 0x7fb3881082a0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.554+0000 7fb37f7fe640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb3740a32d0 0x7fb3740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.554+0000 7fb38659c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb3740a32d0 0x7fb3740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:14.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.554+0000 7fb37f7fe640 1 -- 172.21.15.19:0/2834992817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fb3800cfba0 con 0x7fb3881082a0 2024-09-19T04:06:14.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.554+0000 7fb38659c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb3740a32d0 0x7fb3740a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb378005fd0 tx=0x7fb37800b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:14.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.557+0000 7fb37f7fe640 1 -- 172.21.15.19:0/2834992817 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb38009c420 con 0x7fb3881082a0 2024-09-19T04:06:14.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.739+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb38810c2c0 con 0x7fb3881082a0 2024-09-19T04:06:14.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.739+0000 7fb37f7fe640 1 -- 172.21.15.19:0/2834992817 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb38009bd20 con 0x7fb3881082a0 2024-09-19T04:06:14.741 INFO:teuthology.orchestra.run.smithi019.stdout:244813135920 2024-09-19T04:06:14.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.741+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb3740a32d0 msgr2=0x7fb3740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:14.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.741+0000 7fb38c86c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb3740a32d0 0x7fb3740a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb378005fd0 tx=0x7fb37800b590 comp rx=0 tx=0).stop 2024-09-19T04:06:14.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.741+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881082a0 msgr2=0x7fb38819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:14.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.741+0000 7fb38c86c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881082a0 0x7fb38819a5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb380009fd0 tx=0x7fb38000b260 comp rx=0 tx=0).stop 2024-09-19T04:06:14.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.742+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 shutdown_connections 2024-09-19T04:06:14.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.742+0000 7fb38c86c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb3740a32d0 0x7fb3740a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.742+0000 7fb38c86c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3881082a0 0x7fb38819a5d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.742+0000 7fb38c86c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3881036e0 0x7fb3881a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.742+0000 7fb38c86c640 1 --2- 172.21.15.19:0/2834992817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb388102d30 0x7fb3881a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:14.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.742+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 >> 172.21.15.19:0/2834992817 conn(0x7fb3880fc7d0 msgr2=0x7fb388100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:14.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.742+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 shutdown_connections 2024-09-19T04:06:14.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:14.742+0000 7fb38c86c640 1 -- 172.21.15.19:0/2834992817 wait complete. 2024-09-19T04:06:15.550 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135916 got 244813135920 for osd.8 2024-09-19T04:06:15.551 DEBUG:teuthology.parallel:result is None 2024-09-19T04:06:15.551 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:06:15.551 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:06:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:15.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:15 smithi111 ceph-mon[35365]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:15.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:15 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2834992817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:06:15.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:15 smithi019 ceph-mon[30938]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:15.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:15 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2834992817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:06:15.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:15 smithi196 ceph-mon[35731]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:15.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:15 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2834992817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T04:06:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.270+0000 7f8133a61640 1 -- 172.21.15.19:0/2655160447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c074590 msgr2=0x7f812c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.270+0000 7f8133a61640 1 --2- 172.21.15.19:0/2655160447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c074590 0x7f812c074970 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8128009900 tx=0x7f812802a480 comp rx=0 tx=0).stop 2024-09-19T04:06:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 -- 172.21.15.19:0/2655160447 shutdown_connections 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 --2- 172.21.15.19:0/2655160447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c112b90 0x7f812c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 --2- 172.21.15.19:0/2655160447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f812c074eb0 0x7f812c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 --2- 172.21.15.19:0/2655160447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c074590 0x7f812c074970 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 -- 172.21.15.19:0/2655160447 >> 172.21.15.19:0/2655160447 conn(0x7f812c0fd8c0 msgr2=0x7f812c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 -- 172.21.15.19:0/2655160447 shutdown_connections 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 -- 172.21.15.19:0/2655160447 wait complete. 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 Processor -- start 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 -- start start 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f812c074590 0x7f812c1a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c074eb0 0x7f812c1a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.271+0000 7f8133a61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c112b90 0x7f812c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8133a61640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f812c118570 con 0x7f812c112b90 2024-09-19T04:06:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8133a61640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1183f0 con 0x7f812c074590 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8133a61640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1186f0 con 0x7f812c074eb0 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c112b90 0x7f812c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c112b90 0x7f812c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38420/0 (socket says 172.21.15.19:38420) 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 -- 172.21.15.19:0/3175892584 learned_addr learned my addr 172.21.15.19:0/3175892584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f81317fd640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f812c074590 0x7f812c1a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8130ffc640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c074eb0 0x7f812c1a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 -- 172.21.15.19:0/3175892584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c074eb0 msgr2=0x7f812c1a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c074eb0 0x7f812c1a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 -- 172.21.15.19:0/3175892584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f812c074590 msgr2=0x7f812c1a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f812c074590 0x7f812c1a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 -- 172.21.15.19:0/3175892584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c19f070 con 0x7f812c112b90 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8130ffc640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c074eb0 0x7f812c1a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8131ffe640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c112b90 0x7f812c19e940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f812400e9d0 tx=0x7f812400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f811e7fc640 1 -- 172.21.15.19:0/3175892584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812400cd80 con 0x7f812c112b90 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f811e7fc640 1 -- 172.21.15.19:0/3175892584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8124005020 con 0x7f812c112b90 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812c19f360 con 0x7f812c112b90 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f811e7fc640 1 -- 172.21.15.19:0/3175892584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240116d0 con 0x7f812c112b90 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.272+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812c1ab650 con 0x7f812c112b90 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.273+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812c074970 con 0x7f812c112b90 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.275+0000 7f811e7fc640 1 -- 172.21.15.19:0/3175892584 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f81240051d0 con 0x7f812c112b90 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.275+0000 7f811e7fc640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f81180a32d0 0x7f81180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.275+0000 7f811e7fc640 1 -- 172.21.15.19:0/3175892584 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f81240d0340 con 0x7f812c112b90 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.275+0000 7f81317fd640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f81180a32d0 0x7f81180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.276+0000 7f81317fd640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f81180a32d0 0x7f81180a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f812c19fa80 tx=0x7f81280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.280+0000 7f811e7fc640 1 -- 172.21.15.19:0/3175892584 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8124016030 con 0x7f812c112b90 2024-09-19T04:06:16.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.456+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f812c062630 con 0x7f81180a32d0 2024-09-19T04:06:16.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.465+0000 7f811e7fc640 1 -- 172.21.15.19:0/3175892584 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340690 (secure 0 0 0) 0x7f812c062630 con 0x7f81180a32d0 2024-09-19T04:06:16.469 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f81180a32d0 msgr2=0x7f81180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f81180a32d0 0x7f81180a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f812c19fa80 tx=0x7f81280058d0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c112b90 msgr2=0x7f812c19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c112b90 0x7f812c19e940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f812400e9d0 tx=0x7f812400ee90 comp rx=0 tx=0).stop 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 shutdown_connections 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f81180a32d0 0x7f81180a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c112b90 0x7f812c19e940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c074eb0 0x7f812c1a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 --2- 172.21.15.19:0/3175892584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f812c074590 0x7f812c1a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 >> 172.21.15.19:0/3175892584 conn(0x7f812c0fd8c0 msgr2=0x7f812c0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 shutdown_connections 2024-09-19T04:06:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:16.468+0000 7f8133a61640 1 -- 172.21.15.19:0/3175892584 wait complete. 2024-09-19T04:06:17.236 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-19T04:06:16.433017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659493,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":679,"num_read_kb":460,"num_write":429,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1241696,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349104,"kb_used_data":12300,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592720,"statfs":{"total":1151940427776,"available":1151582945280,"internally_reserved":0,"allocated":12595200,"data_stored":8069309,"data_compressed":398906,"data_compressed_allocated":540672,"data_compressed_original":2613248,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003100"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:11.672149+0000","last_change":"2024-09-19T04:04:37.214769+0000","last_active":"2024-09-19T04:06:11.672149+0000","last_peered":"2024-09-19T04:06:11.672149+0000","last_clean":"2024-09-19T04:06:11.672149+0000","last_became_active":"2024-09-19T04:04:37.214518+0000","last_became_peered":"2024-09-19T04:04:37.214518+0000","last_unstale":"2024-09-19T04:06:11.672149+0000","last_undegraded":"2024-09-19T04:06:11.672149+0000","last_fullsized":"2024-09-19T04:06:11.672149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:50.704566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822826+0000","last_change":"2024-09-19T04:04:39.329733+0000","last_active":"2024-09-19T04:05:42.822826+0000","last_peered":"2024-09-19T04:05:42.822826+0000","last_clean":"2024-09-19T04:05:42.822826+0000","last_became_active":"2024-09-19T04:04:39.329031+0000","last_became_peered":"2024-09-19T04:04:39.329031+0000","last_unstale":"2024-09-19T04:05:42.822826+0000","last_undegraded":"2024-09-19T04:05:42.822826+0000","last_fullsized":"2024-09-19T04:05:42.822826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:57.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.330991+0000","last_change":"2024-09-19T04:04:46.362679+0000","last_active":"2024-09-19T04:05:42.330991+0000","last_peered":"2024-09-19T04:05:42.330991+0000","last_clean":"2024-09-19T04:05:42.330991+0000","last_became_active":"2024-09-19T04:04:46.362257+0000","last_became_peered":"2024-09-19T04:04:46.362257+0000","last_unstale":"2024-09-19T04:05:42.330991+0000","last_undegraded":"2024-09-19T04:05:42.330991+0000","last_fullsized":"2024-09-19T04:05:42.330991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.389839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.823042+0000","last_change":"2024-09-19T04:04:37.647295+0000","last_active":"2024-09-19T04:05:42.823042+0000","last_peered":"2024-09-19T04:05:42.823042+0000","last_clean":"2024-09-19T04:05:42.823042+0000","last_became_active":"2024-09-19T04:04:37.646964+0000","last_became_peered":"2024-09-19T04:04:37.646964+0000","last_unstale":"2024-09-19T04:05:42.823042+0000","last_undegraded":"2024-09-19T04:05:42.823042+0000","last_fullsized":"2024-09-19T04:05:42.823042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:49.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150451+0000","last_change":"2024-09-19T04:04:39.318270+0000","last_active":"2024-09-19T04:05:43.150451+0000","last_peered":"2024-09-19T04:05:43.150451+0000","last_clean":"2024-09-19T04:05:43.150451+0000","last_became_active":"2024-09-19T04:04:39.317972+0000","last_became_peered":"2024-09-19T04:04:39.317972+0000","last_unstale":"2024-09-19T04:05:43.150451+0000","last_undegraded":"2024-09-19T04:05:43.150451+0000","last_fullsized":"2024-09-19T04:05:43.150451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:25.994112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336125+0000","last_change":"2024-09-19T04:04:46.355111+0000","last_active":"2024-09-19T04:05:42.336125+0000","last_peered":"2024-09-19T04:05:42.336125+0000","last_clean":"2024-09-19T04:05:42.336125+0000","last_became_active":"2024-09-19T04:04:46.354684+0000","last_became_peered":"2024-09-19T04:04:46.354684+0000","last_unstale":"2024-09-19T04:05:42.336125+0000","last_undegraded":"2024-09-19T04:05:42.336125+0000","last_fullsized":"2024-09-19T04:05:42.336125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:33.141690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333184+0000","last_change":"2024-09-19T04:04:37.211772+0000","last_active":"2024-09-19T04:05:42.333184+0000","last_peered":"2024-09-19T04:05:42.333184+0000","last_clean":"2024-09-19T04:05:42.333184+0000","last_became_active":"2024-09-19T04:04:37.211452+0000","last_became_peered":"2024-09-19T04:04:37.211452+0000","last_unstale":"2024-09-19T04:05:42.333184+0000","last_undegraded":"2024-09-19T04:05:42.333184+0000","last_fullsized":"2024-09-19T04:05:42.333184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:57.781481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331808+0000","last_change":"2024-09-19T04:04:39.320791+0000","last_active":"2024-09-19T04:05:42.331808+0000","last_peered":"2024-09-19T04:05:42.331808+0000","last_clean":"2024-09-19T04:05:42.331808+0000","last_became_active":"2024-09-19T04:04:39.320206+0000","last_became_peered":"2024-09-19T04:04:39.320206+0000","last_unstale":"2024-09-19T04:05:42.331808+0000","last_undegraded":"2024-09-19T04:05:42.331808+0000","last_fullsized":"2024-09-19T04:05:42.331808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:17.385974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337258+0000","last_change":"2024-09-19T04:04:46.363375+0000","last_active":"2024-09-19T04:05:42.337258+0000","last_peered":"2024-09-19T04:05:42.337258+0000","last_clean":"2024-09-19T04:05:42.337258+0000","last_became_active":"2024-09-19T04:04:46.362800+0000","last_became_peered":"2024-09-19T04:04:46.362800+0000","last_unstale":"2024-09-19T04:05:42.337258+0000","last_undegraded":"2024-09-19T04:05:42.337258+0000","last_fullsized":"2024-09-19T04:05:42.337258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:55.651507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333276+0000","last_change":"2024-09-19T04:04:37.206218+0000","last_active":"2024-09-19T04:05:42.333276+0000","last_peered":"2024-09-19T04:05:42.333276+0000","last_clean":"2024-09-19T04:05:42.333276+0000","last_became_active":"2024-09-19T04:04:37.205675+0000","last_became_peered":"2024-09-19T04:04:37.205675+0000","last_unstale":"2024-09-19T04:05:42.333276+0000","last_undegraded":"2024-09-19T04:05:42.333276+0000","last_fullsized":"2024-09-19T04:05:42.333276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:27.664960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336567+0000","last_change":"2024-09-19T04:04:39.329667+0000","last_active":"2024-09-19T04:05:42.336567+0000","last_peered":"2024-09-19T04:05:42.336567+0000","last_clean":"2024-09-19T04:05:42.336567+0000","last_became_active":"2024-09-19T04:04:39.328180+0000","last_became_peered":"2024-09-19T04:04:39.328180+0000","last_unstale":"2024-09-19T04:05:42.336567+0000","last_undegraded":"2024-09-19T04:05:42.336567+0000","last_fullsized":"2024-09-19T04:05:42.336567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:59.782472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336861+0000","last_change":"2024-09-19T04:04:46.359432+0000","last_active":"2024-09-19T04:05:42.336861+0000","last_peered":"2024-09-19T04:05:42.336861+0000","last_clean":"2024-09-19T04:05:42.336861+0000","last_became_active":"2024-09-19T04:04:46.358966+0000","last_became_peered":"2024-09-19T04:04:46.358966+0000","last_unstale":"2024-09-19T04:05:42.336861+0000","last_undegraded":"2024-09-19T04:05:42.336861+0000","last_fullsized":"2024-09-19T04:05:42.336861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:41.537926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337468+0000","last_change":"2024-09-19T04:04:39.325084+0000","last_active":"2024-09-19T04:05:42.337468+0000","last_peered":"2024-09-19T04:05:42.337468+0000","last_clean":"2024-09-19T04:05:42.337468+0000","last_became_active":"2024-09-19T04:04:39.324126+0000","last_became_peered":"2024-09-19T04:04:39.324126+0000","last_unstale":"2024-09-19T04:05:42.337468+0000","last_undegraded":"2024-09-19T04:05:42.337468+0000","last_fullsized":"2024-09-19T04:05:42.337468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:13.523889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.359505+0000","last_change":"2024-09-19T04:05:42.359098+0000","last_active":"2024-09-19T04:05:42.359505+0000","last_peered":"2024-09-19T04:05:42.359505+0000","last_clean":"2024-09-19T04:05:42.359505+0000","last_became_active":"2024-09-19T04:05:42.341384+0000","last_became_peered":"2024-09-19T04:05:42.341384+0000","last_unstale":"2024-09-19T04:05:42.359505+0000","last_undegraded":"2024-09-19T04:05:42.359505+0000","last_fullsized":"2024-09-19T04:05:42.359505+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:15:57.418355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336594+0000","last_change":"2024-09-19T04:04:46.374921+0000","last_active":"2024-09-19T04:05:42.336594+0000","last_peered":"2024-09-19T04:05:42.336594+0000","last_clean":"2024-09-19T04:05:42.336594+0000","last_became_active":"2024-09-19T04:04:46.374466+0000","last_became_peered":"2024-09-19T04:04:46.374466+0000","last_unstale":"2024-09-19T04:05:42.336594+0000","last_undegraded":"2024-09-19T04:05:42.336594+0000","last_fullsized":"2024-09-19T04:05:42.336594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:46.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822761+0000","last_change":"2024-09-19T04:04:39.319948+0000","last_active":"2024-09-19T04:05:42.822761+0000","last_peered":"2024-09-19T04:05:42.822761+0000","last_clean":"2024-09-19T04:05:42.822761+0000","last_became_active":"2024-09-19T04:04:39.319675+0000","last_became_peered":"2024-09-19T04:04:39.319675+0000","last_unstale":"2024-09-19T04:05:42.822761+0000","last_undegraded":"2024-09-19T04:05:42.822761+0000","last_fullsized":"2024-09-19T04:05:42.822761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:08.818133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331510+0000","last_change":"2024-09-19T04:04:37.207705+0000","last_active":"2024-09-19T04:05:42.331510+0000","last_peered":"2024-09-19T04:05:42.331510+0000","last_clean":"2024-09-19T04:05:42.331510+0000","last_became_active":"2024-09-19T04:04:37.207251+0000","last_became_peered":"2024-09-19T04:04:37.207251+0000","last_unstale":"2024-09-19T04:05:42.331510+0000","last_undegraded":"2024-09-19T04:05:42.331510+0000","last_fullsized":"2024-09-19T04:05:42.331510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:20.835004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336728+0000","last_change":"2024-09-19T04:04:46.354463+0000","last_active":"2024-09-19T04:05:42.336728+0000","last_peered":"2024-09-19T04:05:42.336728+0000","last_clean":"2024-09-19T04:05:42.336728+0000","last_became_active":"2024-09-19T04:04:46.353754+0000","last_became_peered":"2024-09-19T04:04:46.353754+0000","last_unstale":"2024-09-19T04:05:42.336728+0000","last_undegraded":"2024-09-19T04:05:42.336728+0000","last_fullsized":"2024-09-19T04:05:42.336728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:09.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333676+0000","last_change":"2024-09-19T04:04:39.322727+0000","last_active":"2024-09-19T04:05:42.333676+0000","last_peered":"2024-09-19T04:05:42.333676+0000","last_clean":"2024-09-19T04:05:42.333676+0000","last_became_active":"2024-09-19T04:04:39.320989+0000","last_became_peered":"2024-09-19T04:04:39.320989+0000","last_unstale":"2024-09-19T04:05:42.333676+0000","last_undegraded":"2024-09-19T04:05:42.333676+0000","last_fullsized":"2024-09-19T04:05:42.333676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:06.536540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822964+0000","last_change":"2024-09-19T04:04:37.213602+0000","last_active":"2024-09-19T04:05:42.822964+0000","last_peered":"2024-09-19T04:05:42.822964+0000","last_clean":"2024-09-19T04:05:42.822964+0000","last_became_active":"2024-09-19T04:04:37.213240+0000","last_became_peered":"2024-09-19T04:04:37.213240+0000","last_unstale":"2024-09-19T04:05:42.822964+0000","last_undegraded":"2024-09-19T04:05:42.822964+0000","last_fullsized":"2024-09-19T04:05:42.822964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:09.822393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333269+0000","last_change":"2024-09-19T04:04:46.363921+0000","last_active":"2024-09-19T04:05:42.333269+0000","last_peered":"2024-09-19T04:05:42.333269+0000","last_clean":"2024-09-19T04:05:42.333269+0000","last_became_active":"2024-09-19T04:04:46.363476+0000","last_became_peered":"2024-09-19T04:04:46.363476+0000","last_unstale":"2024-09-19T04:05:42.333269+0000","last_undegraded":"2024-09-19T04:05:42.333269+0000","last_fullsized":"2024-09-19T04:05:42.333269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:32.015782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332715+0000","last_change":"2024-09-19T04:04:39.316814+0000","last_active":"2024-09-19T04:05:42.332715+0000","last_peered":"2024-09-19T04:05:42.332715+0000","last_clean":"2024-09-19T04:05:42.332715+0000","last_became_active":"2024-09-19T04:04:39.316504+0000","last_became_peered":"2024-09-19T04:04:39.316504+0000","last_unstale":"2024-09-19T04:05:42.332715+0000","last_undegraded":"2024-09-19T04:05:42.332715+0000","last_fullsized":"2024-09-19T04:05:42.332715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331734+0000","last_change":"2024-09-19T04:04:37.647260+0000","last_active":"2024-09-19T04:05:42.331734+0000","last_peered":"2024-09-19T04:05:42.331734+0000","last_clean":"2024-09-19T04:05:42.331734+0000","last_became_active":"2024-09-19T04:04:37.646971+0000","last_became_peered":"2024-09-19T04:04:37.646971+0000","last_unstale":"2024-09-19T04:05:42.331734+0000","last_undegraded":"2024-09-19T04:05:42.331734+0000","last_fullsized":"2024-09-19T04:05:42.331734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:07.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.342340+0000","last_change":"2024-09-19T04:05:42.342340+0000","last_active":"2024-09-19T04:05:42.342340+0000","last_peered":"2024-09-19T04:05:42.342340+0000","last_clean":"2024-09-19T04:05:42.342340+0000","last_became_active":"2024-09-19T04:05:42.341861+0000","last_became_peered":"2024-09-19T04:05:42.341861+0000","last_unstale":"2024-09-19T04:05:42.342340+0000","last_undegraded":"2024-09-19T04:05:42.342340+0000","last_fullsized":"2024-09-19T04:05:42.342340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:27.520514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150301+0000","last_change":"2024-09-19T04:04:39.318185+0000","last_active":"2024-09-19T04:05:43.150301+0000","last_peered":"2024-09-19T04:05:43.150301+0000","last_clean":"2024-09-19T04:05:43.150301+0000","last_became_active":"2024-09-19T04:04:39.317885+0000","last_became_peered":"2024-09-19T04:04:39.317885+0000","last_unstale":"2024-09-19T04:05:43.150301+0000","last_undegraded":"2024-09-19T04:05:43.150301+0000","last_fullsized":"2024-09-19T04:05:43.150301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:27.719248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337062+0000","last_change":"2024-09-19T04:04:37.703740+0000","last_active":"2024-09-19T04:05:42.337062+0000","last_peered":"2024-09-19T04:05:42.337062+0000","last_clean":"2024-09-19T04:05:42.337062+0000","last_became_active":"2024-09-19T04:04:37.703414+0000","last_became_peered":"2024-09-19T04:04:37.703414+0000","last_unstale":"2024-09-19T04:05:42.337062+0000","last_undegraded":"2024-09-19T04:05:42.337062+0000","last_fullsized":"2024-09-19T04:05:42.337062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:19.653559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331302+0000","last_change":"2024-09-19T04:04:46.354964+0000","last_active":"2024-09-19T04:05:42.331302+0000","last_peered":"2024-09-19T04:05:42.331302+0000","last_clean":"2024-09-19T04:05:42.331302+0000","last_became_active":"2024-09-19T04:04:46.354494+0000","last_became_peered":"2024-09-19T04:04:46.354494+0000","last_unstale":"2024-09-19T04:05:42.331302+0000","last_undegraded":"2024-09-19T04:05:42.331302+0000","last_fullsized":"2024-09-19T04:05:42.331302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:11.246134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.823151+0000","last_change":"2024-09-19T04:04:39.315811+0000","last_active":"2024-09-19T04:05:42.823151+0000","last_peered":"2024-09-19T04:05:42.823151+0000","last_clean":"2024-09-19T04:05:42.823151+0000","last_became_active":"2024-09-19T04:04:39.315466+0000","last_became_peered":"2024-09-19T04:04:39.315466+0000","last_unstale":"2024-09-19T04:05:42.823151+0000","last_undegraded":"2024-09-19T04:05:42.823151+0000","last_fullsized":"2024-09-19T04:05:42.823151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:17.414765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333299+0000","last_change":"2024-09-19T04:04:37.204206+0000","last_active":"2024-09-19T04:05:42.333299+0000","last_peered":"2024-09-19T04:05:42.333299+0000","last_clean":"2024-09-19T04:05:42.333299+0000","last_became_active":"2024-09-19T04:04:37.203845+0000","last_became_peered":"2024-09-19T04:04:37.203845+0000","last_unstale":"2024-09-19T04:05:42.333299+0000","last_undegraded":"2024-09-19T04:05:42.333299+0000","last_fullsized":"2024-09-19T04:05:42.333299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:52.868643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822240+0000","last_change":"2024-09-19T04:04:46.375541+0000","last_active":"2024-09-19T04:05:42.822240+0000","last_peered":"2024-09-19T04:05:42.822240+0000","last_clean":"2024-09-19T04:05:42.822240+0000","last_became_active":"2024-09-19T04:04:46.375142+0000","last_became_peered":"2024-09-19T04:04:46.375142+0000","last_unstale":"2024-09-19T04:05:42.822240+0000","last_undegraded":"2024-09-19T04:05:42.822240+0000","last_fullsized":"2024-09-19T04:05:42.822240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.889459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464943+0000","last_change":"2024-09-19T04:04:39.324094+0000","last_active":"2024-09-19T04:05:42.464943+0000","last_peered":"2024-09-19T04:05:42.464943+0000","last_clean":"2024-09-19T04:05:42.464943+0000","last_became_active":"2024-09-19T04:04:39.322897+0000","last_became_peered":"2024-09-19T04:04:39.322897+0000","last_unstale":"2024-09-19T04:05:42.464943+0000","last_undegraded":"2024-09-19T04:05:42.464943+0000","last_fullsized":"2024-09-19T04:05:42.464943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:07.773110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333314+0000","last_change":"2024-09-19T04:04:37.647686+0000","last_active":"2024-09-19T04:05:42.333314+0000","last_peered":"2024-09-19T04:05:42.333314+0000","last_clean":"2024-09-19T04:05:42.333314+0000","last_became_active":"2024-09-19T04:04:37.646979+0000","last_became_peered":"2024-09-19T04:04:37.646979+0000","last_unstale":"2024-09-19T04:05:42.333314+0000","last_undegraded":"2024-09-19T04:05:42.333314+0000","last_fullsized":"2024-09-19T04:05:42.333314+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:48.063075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335982+0000","last_change":"2024-09-19T04:04:46.356078+0000","last_active":"2024-09-19T04:05:42.335982+0000","last_peered":"2024-09-19T04:05:42.335982+0000","last_clean":"2024-09-19T04:05:42.335982+0000","last_became_active":"2024-09-19T04:04:46.355630+0000","last_became_peered":"2024-09-19T04:04:46.355630+0000","last_unstale":"2024-09-19T04:05:42.335982+0000","last_undegraded":"2024-09-19T04:05:42.335982+0000","last_fullsized":"2024-09-19T04:05:42.335982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:45.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334352+0000","last_change":"2024-09-19T04:04:39.311067+0000","last_active":"2024-09-19T04:05:42.334352+0000","last_peered":"2024-09-19T04:05:42.334352+0000","last_clean":"2024-09-19T04:05:42.334352+0000","last_became_active":"2024-09-19T04:04:39.310725+0000","last_became_peered":"2024-09-19T04:04:39.310725+0000","last_unstale":"2024-09-19T04:05:42.334352+0000","last_undegraded":"2024-09-19T04:05:42.334352+0000","last_fullsized":"2024-09-19T04:05:42.334352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:47:11.488121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333056+0000","last_change":"2024-09-19T04:04:37.648335+0000","last_active":"2024-09-19T04:05:42.333056+0000","last_peered":"2024-09-19T04:05:42.333056+0000","last_clean":"2024-09-19T04:05:42.333056+0000","last_became_active":"2024-09-19T04:04:37.648000+0000","last_became_peered":"2024-09-19T04:04:37.648000+0000","last_unstale":"2024-09-19T04:05:42.333056+0000","last_undegraded":"2024-09-19T04:05:42.333056+0000","last_fullsized":"2024-09-19T04:05:42.333056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:16.102728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332783+0000","last_change":"2024-09-19T04:04:46.375653+0000","last_active":"2024-09-19T04:05:42.332783+0000","last_peered":"2024-09-19T04:05:42.332783+0000","last_clean":"2024-09-19T04:05:42.332783+0000","last_became_active":"2024-09-19T04:04:46.375132+0000","last_became_peered":"2024-09-19T04:04:46.375132+0000","last_unstale":"2024-09-19T04:05:42.332783+0000","last_undegraded":"2024-09-19T04:05:42.332783+0000","last_fullsized":"2024-09-19T04:05:42.332783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:03.332697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334076+0000","last_change":"2024-09-19T04:04:39.318155+0000","last_active":"2024-09-19T04:05:42.334076+0000","last_peered":"2024-09-19T04:05:42.334076+0000","last_clean":"2024-09-19T04:05:42.334076+0000","last_became_active":"2024-09-19T04:04:39.316710+0000","last_became_peered":"2024-09-19T04:04:39.316710+0000","last_unstale":"2024-09-19T04:05:42.334076+0000","last_undegraded":"2024-09-19T04:05:42.334076+0000","last_fullsized":"2024-09-19T04:05:42.334076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:59.084523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":95,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:13.675552+0000","last_change":"2024-09-19T04:04:37.210695+0000","last_active":"2024-09-19T04:06:13.675552+0000","last_peered":"2024-09-19T04:06:13.675552+0000","last_clean":"2024-09-19T04:06:13.675552+0000","last_became_active":"2024-09-19T04:04:37.210334+0000","last_became_peered":"2024-09-19T04:04:37.210334+0000","last_unstale":"2024-09-19T04:06:13.675552+0000","last_undegraded":"2024-09-19T04:06:13.675552+0000","last_fullsized":"2024-09-19T04:06:13.675552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:12.545248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464242+0000","last_change":"2024-09-19T04:04:46.358130+0000","last_active":"2024-09-19T04:05:42.464242+0000","last_peered":"2024-09-19T04:05:42.464242+0000","last_clean":"2024-09-19T04:05:42.464242+0000","last_became_active":"2024-09-19T04:04:46.356810+0000","last_became_peered":"2024-09-19T04:04:46.356810+0000","last_unstale":"2024-09-19T04:05:42.464242+0000","last_undegraded":"2024-09-19T04:05:42.464242+0000","last_fullsized":"2024-09-19T04:05:42.464242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:49.884478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335465+0000","last_change":"2024-09-19T04:04:39.325866+0000","last_active":"2024-09-19T04:05:42.335465+0000","last_peered":"2024-09-19T04:05:42.335465+0000","last_clean":"2024-09-19T04:05:42.335465+0000","last_became_active":"2024-09-19T04:04:39.325473+0000","last_became_peered":"2024-09-19T04:04:39.325473+0000","last_unstale":"2024-09-19T04:05:42.335465+0000","last_undegraded":"2024-09-19T04:05:42.335465+0000","last_fullsized":"2024-09-19T04:05:42.335465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:30.036704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:14.676349+0000","last_change":"2024-09-19T04:04:37.647310+0000","last_active":"2024-09-19T04:06:14.676349+0000","last_peered":"2024-09-19T04:06:14.676349+0000","last_clean":"2024-09-19T04:06:14.676349+0000","last_became_active":"2024-09-19T04:04:37.646903+0000","last_became_peered":"2024-09-19T04:04:37.646903+0000","last_unstale":"2024-09-19T04:06:14.676349+0000","last_undegraded":"2024-09-19T04:06:14.676349+0000","last_fullsized":"2024-09-19T04:06:14.676349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:20.097472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464406+0000","last_change":"2024-09-19T04:04:46.375261+0000","last_active":"2024-09-19T04:05:42.464406+0000","last_peered":"2024-09-19T04:05:42.464406+0000","last_clean":"2024-09-19T04:05:42.464406+0000","last_became_active":"2024-09-19T04:04:46.374489+0000","last_became_peered":"2024-09-19T04:04:46.374489+0000","last_unstale":"2024-09-19T04:05:42.464406+0000","last_undegraded":"2024-09-19T04:05:42.464406+0000","last_fullsized":"2024-09-19T04:05:42.464406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:05.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464375+0000","last_change":"2024-09-19T04:04:39.322265+0000","last_active":"2024-09-19T04:05:42.464375+0000","last_peered":"2024-09-19T04:05:42.464375+0000","last_clean":"2024-09-19T04:05:42.464375+0000","last_became_active":"2024-09-19T04:04:39.321938+0000","last_became_peered":"2024-09-19T04:04:39.321938+0000","last_unstale":"2024-09-19T04:05:42.464375+0000","last_undegraded":"2024-09-19T04:05:42.464375+0000","last_fullsized":"2024-09-19T04:05:42.464375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:19.220661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.339577+0000","last_change":"2024-09-19T04:04:37.211330+0000","last_active":"2024-09-19T04:05:42.339577+0000","last_peered":"2024-09-19T04:05:42.339577+0000","last_clean":"2024-09-19T04:05:42.339577+0000","last_became_active":"2024-09-19T04:04:37.210788+0000","last_became_peered":"2024-09-19T04:04:37.210788+0000","last_unstale":"2024-09-19T04:05:42.339577+0000","last_undegraded":"2024-09-19T04:05:42.339577+0000","last_fullsized":"2024-09-19T04:05:42.339577+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:43.169897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.465969+0000","last_change":"2024-09-19T04:05:42.465969+0000","last_active":"2024-09-19T04:05:42.465969+0000","last_peered":"2024-09-19T04:05:42.465969+0000","last_clean":"2024-09-19T04:05:42.465969+0000","last_became_active":"2024-09-19T04:05:42.465621+0000","last_became_peered":"2024-09-19T04:05:42.465621+0000","last_unstale":"2024-09-19T04:05:42.465969+0000","last_undegraded":"2024-09-19T04:05:42.465969+0000","last_fullsized":"2024-09-19T04:05:42.465969+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:29.802141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,10],"acting":[5,1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.823049+0000","last_change":"2024-09-19T04:04:39.329480+0000","last_active":"2024-09-19T04:05:42.823049+0000","last_peered":"2024-09-19T04:05:42.823049+0000","last_clean":"2024-09-19T04:05:42.823049+0000","last_became_active":"2024-09-19T04:04:39.329008+0000","last_became_peered":"2024-09-19T04:04:39.329008+0000","last_unstale":"2024-09-19T04:05:42.823049+0000","last_undegraded":"2024-09-19T04:05:42.823049+0000","last_fullsized":"2024-09-19T04:05:42.823049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:06.782579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337719+0000","last_change":"2024-09-19T04:04:37.216224+0000","last_active":"2024-09-19T04:05:42.337719+0000","last_peered":"2024-09-19T04:05:42.337719+0000","last_clean":"2024-09-19T04:05:42.337719+0000","last_became_active":"2024-09-19T04:04:37.215846+0000","last_became_peered":"2024-09-19T04:04:37.215846+0000","last_unstale":"2024-09-19T04:05:42.337719+0000","last_undegraded":"2024-09-19T04:05:42.337719+0000","last_fullsized":"2024-09-19T04:05:42.337719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:57.829694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.339830+0000","last_change":"2024-09-19T04:03:25.869634+0000","last_active":"2024-09-19T04:05:42.339830+0000","last_peered":"2024-09-19T04:05:42.339830+0000","last_clean":"2024-09-19T04:05:42.339830+0000","last_became_active":"2024-09-19T04:03:25.869250+0000","last_became_peered":"2024-09-19T04:03:25.869250+0000","last_unstale":"2024-09-19T04:05:42.339830+0000","last_undegraded":"2024-09-19T04:05:42.339830+0000","last_fullsized":"2024-09-19T04:05:42.339830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_clean_scrub_stamp":"2024-09-19T03:59:59.253417+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:51.956932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338108+0000","last_change":"2024-09-19T04:04:46.357123+0000","last_active":"2024-09-19T04:05:42.338108+0000","last_peered":"2024-09-19T04:05:42.338108+0000","last_clean":"2024-09-19T04:05:42.338108+0000","last_became_active":"2024-09-19T04:04:46.356671+0000","last_became_peered":"2024-09-19T04:04:46.356671+0000","last_unstale":"2024-09-19T04:05:42.338108+0000","last_undegraded":"2024-09-19T04:05:42.338108+0000","last_fullsized":"2024-09-19T04:05:42.338108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:09.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338394+0000","last_change":"2024-09-19T04:04:39.322487+0000","last_active":"2024-09-19T04:05:42.338394+0000","last_peered":"2024-09-19T04:05:42.338394+0000","last_clean":"2024-09-19T04:05:42.338394+0000","last_became_active":"2024-09-19T04:04:39.321994+0000","last_became_peered":"2024-09-19T04:04:39.321994+0000","last_unstale":"2024-09-19T04:05:42.338394+0000","last_undegraded":"2024-09-19T04:05:42.338394+0000","last_fullsized":"2024-09-19T04:05:42.338394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:56.977405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:11.676946+0000","last_change":"2024-09-19T04:04:37.703452+0000","last_active":"2024-09-19T04:06:11.676946+0000","last_peered":"2024-09-19T04:06:11.676946+0000","last_clean":"2024-09-19T04:06:11.676946+0000","last_became_active":"2024-09-19T04:04:37.703048+0000","last_became_peered":"2024-09-19T04:04:37.703048+0000","last_unstale":"2024-09-19T04:06:11.676946+0000","last_undegraded":"2024-09-19T04:06:11.676946+0000","last_fullsized":"2024-09-19T04:06:11.676946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:16:29.051617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331703+0000","last_change":"2024-09-19T04:04:46.361229+0000","last_active":"2024-09-19T04:05:42.331703+0000","last_peered":"2024-09-19T04:05:42.331703+0000","last_clean":"2024-09-19T04:05:42.331703+0000","last_became_active":"2024-09-19T04:04:46.360729+0000","last_became_peered":"2024-09-19T04:04:46.360729+0000","last_unstale":"2024-09-19T04:05:42.331703+0000","last_undegraded":"2024-09-19T04:05:42.331703+0000","last_fullsized":"2024-09-19T04:05:42.331703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:48.022778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337919+0000","last_change":"2024-09-19T04:04:39.328598+0000","last_active":"2024-09-19T04:05:42.337919+0000","last_peered":"2024-09-19T04:05:42.337919+0000","last_clean":"2024-09-19T04:05:42.337919+0000","last_became_active":"2024-09-19T04:04:39.328247+0000","last_became_peered":"2024-09-19T04:04:39.328247+0000","last_unstale":"2024-09-19T04:05:42.337919+0000","last_undegraded":"2024-09-19T04:05:42.337919+0000","last_fullsized":"2024-09-19T04:05:42.337919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:34.916487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336864+0000","last_change":"2024-09-19T04:04:37.212065+0000","last_active":"2024-09-19T04:05:42.336864+0000","last_peered":"2024-09-19T04:05:42.336864+0000","last_clean":"2024-09-19T04:05:42.336864+0000","last_became_active":"2024-09-19T04:04:37.211520+0000","last_became_peered":"2024-09-19T04:04:37.211520+0000","last_unstale":"2024-09-19T04:05:42.336864+0000","last_undegraded":"2024-09-19T04:05:42.336864+0000","last_fullsized":"2024-09-19T04:05:42.336864+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:53.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.149487+0000","last_change":"2024-09-19T04:04:46.358558+0000","last_active":"2024-09-19T04:05:43.149487+0000","last_peered":"2024-09-19T04:05:43.149487+0000","last_clean":"2024-09-19T04:05:43.149487+0000","last_became_active":"2024-09-19T04:04:46.358158+0000","last_became_peered":"2024-09-19T04:04:46.358158+0000","last_unstale":"2024-09-19T04:05:43.149487+0000","last_undegraded":"2024-09-19T04:05:43.149487+0000","last_fullsized":"2024-09-19T04:05:43.149487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:16.627051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338837+0000","last_change":"2024-09-19T04:04:39.325187+0000","last_active":"2024-09-19T04:05:42.338837+0000","last_peered":"2024-09-19T04:05:42.338837+0000","last_clean":"2024-09-19T04:05:42.338837+0000","last_became_active":"2024-09-19T04:04:39.324644+0000","last_became_peered":"2024-09-19T04:04:39.324644+0000","last_unstale":"2024-09-19T04:05:42.338837+0000","last_undegraded":"2024-09-19T04:05:42.338837+0000","last_fullsized":"2024-09-19T04:05:42.338837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:11.879273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464944+0000","last_change":"2024-09-19T04:04:37.205389+0000","last_active":"2024-09-19T04:05:42.464944+0000","last_peered":"2024-09-19T04:05:42.464944+0000","last_clean":"2024-09-19T04:05:42.464944+0000","last_became_active":"2024-09-19T04:04:37.205117+0000","last_became_peered":"2024-09-19T04:04:37.205117+0000","last_unstale":"2024-09-19T04:05:42.464944+0000","last_undegraded":"2024-09-19T04:05:42.464944+0000","last_fullsized":"2024-09-19T04:05:42.464944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:49.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.342246+0000","last_change":"2024-09-19T04:05:42.342246+0000","last_active":"2024-09-19T04:05:42.342246+0000","last_peered":"2024-09-19T04:05:42.342246+0000","last_clean":"2024-09-19T04:05:42.342246+0000","last_became_active":"2024-09-19T04:05:42.341628+0000","last_became_peered":"2024-09-19T04:05:42.341628+0000","last_unstale":"2024-09-19T04:05:42.342246+0000","last_undegraded":"2024-09-19T04:05:42.342246+0000","last_fullsized":"2024-09-19T04:05:42.342246+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:16.637647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333643+0000","last_change":"2024-09-19T04:04:39.328841+0000","last_active":"2024-09-19T04:05:42.333643+0000","last_peered":"2024-09-19T04:05:42.333643+0000","last_clean":"2024-09-19T04:05:42.333643+0000","last_became_active":"2024-09-19T04:04:39.328470+0000","last_became_peered":"2024-09-19T04:04:39.328470+0000","last_unstale":"2024-09-19T04:05:42.333643+0000","last_undegraded":"2024-09-19T04:05:42.333643+0000","last_fullsized":"2024-09-19T04:05:42.333643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:17.701469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.353928+0000","last_change":"2024-09-19T04:05:42.353456+0000","last_active":"2024-09-19T04:05:42.353928+0000","last_peered":"2024-09-19T04:05:42.353928+0000","last_clean":"2024-09-19T04:05:42.353928+0000","last_became_active":"2024-09-19T04:05:42.340710+0000","last_became_peered":"2024-09-19T04:05:42.340710+0000","last_unstale":"2024-09-19T04:05:42.353928+0000","last_undegraded":"2024-09-19T04:05:42.353928+0000","last_fullsized":"2024-09-19T04:05:42.353928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:45.262693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335416+0000","last_change":"2024-09-19T04:04:46.361199+0000","last_active":"2024-09-19T04:05:42.335416+0000","last_peered":"2024-09-19T04:05:42.335416+0000","last_clean":"2024-09-19T04:05:42.335416+0000","last_became_active":"2024-09-19T04:04:46.360639+0000","last_became_peered":"2024-09-19T04:04:46.360639+0000","last_unstale":"2024-09-19T04:05:42.335416+0000","last_undegraded":"2024-09-19T04:05:42.335416+0000","last_fullsized":"2024-09-19T04:05:42.335416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:50.185760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335552+0000","last_change":"2024-09-19T04:04:39.328529+0000","last_active":"2024-09-19T04:05:42.335552+0000","last_peered":"2024-09-19T04:05:42.335552+0000","last_clean":"2024-09-19T04:05:42.335552+0000","last_became_active":"2024-09-19T04:04:39.328174+0000","last_became_peered":"2024-09-19T04:04:39.328174+0000","last_unstale":"2024-09-19T04:05:42.335552+0000","last_undegraded":"2024-09-19T04:05:42.335552+0000","last_fullsized":"2024-09-19T04:05:42.335552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:02.503672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335126+0000","last_change":"2024-09-19T04:04:37.647919+0000","last_active":"2024-09-19T04:05:42.335126+0000","last_peered":"2024-09-19T04:05:42.335126+0000","last_clean":"2024-09-19T04:05:42.335126+0000","last_became_active":"2024-09-19T04:04:37.647305+0000","last_became_peered":"2024-09-19T04:04:37.647305+0000","last_unstale":"2024-09-19T04:05:42.335126+0000","last_undegraded":"2024-09-19T04:05:42.335126+0000","last_fullsized":"2024-09-19T04:05:42.335126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:07.035465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464291+0000","last_change":"2024-09-19T04:04:46.360362+0000","last_active":"2024-09-19T04:05:42.464291+0000","last_peered":"2024-09-19T04:05:42.464291+0000","last_clean":"2024-09-19T04:05:42.464291+0000","last_became_active":"2024-09-19T04:04:46.359470+0000","last_became_peered":"2024-09-19T04:04:46.359470+0000","last_unstale":"2024-09-19T04:05:42.464291+0000","last_undegraded":"2024-09-19T04:05:42.464291+0000","last_fullsized":"2024-09-19T04:05:42.464291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:43.804073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.339402+0000","last_change":"2024-09-19T04:04:39.326090+0000","last_active":"2024-09-19T04:05:42.339402+0000","last_peered":"2024-09-19T04:05:42.339402+0000","last_clean":"2024-09-19T04:05:42.339402+0000","last_became_active":"2024-09-19T04:04:39.325751+0000","last_became_peered":"2024-09-19T04:04:39.325751+0000","last_unstale":"2024-09-19T04:05:42.339402+0000","last_undegraded":"2024-09-19T04:05:42.339402+0000","last_fullsized":"2024-09-19T04:05:42.339402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:25.357467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:13.675638+0000","last_change":"2024-09-19T04:04:37.215107+0000","last_active":"2024-09-19T04:06:13.675638+0000","last_peered":"2024-09-19T04:06:13.675638+0000","last_clean":"2024-09-19T04:06:13.675638+0000","last_became_active":"2024-09-19T04:04:37.214693+0000","last_became_peered":"2024-09-19T04:04:37.214693+0000","last_unstale":"2024-09-19T04:06:13.675638+0000","last_undegraded":"2024-09-19T04:06:13.675638+0000","last_fullsized":"2024-09-19T04:06:13.675638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:56.535316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333298+0000","last_change":"2024-09-19T04:04:46.363625+0000","last_active":"2024-09-19T04:05:42.333298+0000","last_peered":"2024-09-19T04:05:42.333298+0000","last_clean":"2024-09-19T04:05:42.333298+0000","last_became_active":"2024-09-19T04:04:46.363207+0000","last_became_peered":"2024-09-19T04:04:46.363207+0000","last_unstale":"2024-09-19T04:05:42.333298+0000","last_undegraded":"2024-09-19T04:05:42.333298+0000","last_fullsized":"2024-09-19T04:05:42.333298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:30.251168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.466411+0000","last_change":"2024-09-19T04:05:42.466411+0000","last_active":"2024-09-19T04:05:42.466411+0000","last_peered":"2024-09-19T04:05:42.466411+0000","last_clean":"2024-09-19T04:05:42.466411+0000","last_became_active":"2024-09-19T04:05:42.465999+0000","last_became_peered":"2024-09-19T04:05:42.465999+0000","last_unstale":"2024-09-19T04:05:42.466411+0000","last_undegraded":"2024-09-19T04:05:42.466411+0000","last_fullsized":"2024-09-19T04:05:42.466411+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:51.289726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336089+0000","last_change":"2024-09-19T04:04:37.212103+0000","last_active":"2024-09-19T04:05:42.336089+0000","last_peered":"2024-09-19T04:05:42.336089+0000","last_clean":"2024-09-19T04:05:42.336089+0000","last_became_active":"2024-09-19T04:04:37.211739+0000","last_became_peered":"2024-09-19T04:04:37.211739+0000","last_unstale":"2024-09-19T04:05:42.336089+0000","last_undegraded":"2024-09-19T04:05:42.336089+0000","last_fullsized":"2024-09-19T04:05:42.336089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:06.588194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333377+0000","last_change":"2024-09-19T04:04:46.361975+0000","last_active":"2024-09-19T04:05:42.333377+0000","last_peered":"2024-09-19T04:05:42.333377+0000","last_clean":"2024-09-19T04:05:42.333377+0000","last_became_active":"2024-09-19T04:04:46.361492+0000","last_became_peered":"2024-09-19T04:04:46.361492+0000","last_unstale":"2024-09-19T04:05:42.333377+0000","last_undegraded":"2024-09-19T04:05:42.333377+0000","last_fullsized":"2024-09-19T04:05:42.333377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:52.723836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464862+0000","last_change":"2024-09-19T04:04:39.317756+0000","last_active":"2024-09-19T04:05:42.464862+0000","last_peered":"2024-09-19T04:05:42.464862+0000","last_clean":"2024-09-19T04:05:42.464862+0000","last_became_active":"2024-09-19T04:04:39.317322+0000","last_became_peered":"2024-09-19T04:04:39.317322+0000","last_unstale":"2024-09-19T04:05:42.464862+0000","last_undegraded":"2024-09-19T04:05:42.464862+0000","last_fullsized":"2024-09-19T04:05:42.464862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:26.291574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331895+0000","last_change":"2024-09-19T04:04:37.213892+0000","last_active":"2024-09-19T04:05:42.331895+0000","last_peered":"2024-09-19T04:05:42.331895+0000","last_clean":"2024-09-19T04:05:42.331895+0000","last_became_active":"2024-09-19T04:04:37.213350+0000","last_became_peered":"2024-09-19T04:04:37.213350+0000","last_unstale":"2024-09-19T04:05:42.331895+0000","last_undegraded":"2024-09-19T04:05:42.331895+0000","last_fullsized":"2024-09-19T04:05:42.331895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:40.065717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336452+0000","last_change":"2024-09-19T04:04:46.360566+0000","last_active":"2024-09-19T04:05:42.336452+0000","last_peered":"2024-09-19T04:05:42.336452+0000","last_clean":"2024-09-19T04:05:42.336452+0000","last_became_active":"2024-09-19T04:04:46.360143+0000","last_became_peered":"2024-09-19T04:04:46.360143+0000","last_unstale":"2024-09-19T04:05:42.336452+0000","last_undegraded":"2024-09-19T04:05:42.336452+0000","last_fullsized":"2024-09-19T04:05:42.336452+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:59.997088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332966+0000","last_change":"2024-09-19T04:04:39.318363+0000","last_active":"2024-09-19T04:05:42.332966+0000","last_peered":"2024-09-19T04:05:42.332966+0000","last_clean":"2024-09-19T04:05:42.332966+0000","last_became_active":"2024-09-19T04:04:39.318003+0000","last_became_peered":"2024-09-19T04:04:39.318003+0000","last_unstale":"2024-09-19T04:05:42.332966+0000","last_undegraded":"2024-09-19T04:05:42.332966+0000","last_fullsized":"2024-09-19T04:05:42.332966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:54.128795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":196,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:12.385843+0000","last_change":"2024-09-19T04:05:42.349944+0000","last_active":"2024-09-19T04:06:12.385843+0000","last_peered":"2024-09-19T04:06:12.385843+0000","last_clean":"2024-09-19T04:06:12.385843+0000","last_became_active":"2024-09-19T04:05:42.339605+0000","last_became_peered":"2024-09-19T04:05:42.339605+0000","last_unstale":"2024-09-19T04:06:12.385843+0000","last_undegraded":"2024-09-19T04:06:12.385843+0000","last_fullsized":"2024-09-19T04:06:12.385843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:17.508316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":116,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":60938,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332887+0000","last_change":"2024-09-19T04:04:46.362967+0000","last_active":"2024-09-19T04:05:42.332887+0000","last_peered":"2024-09-19T04:05:42.332887+0000","last_clean":"2024-09-19T04:05:42.332887+0000","last_became_active":"2024-09-19T04:04:46.362529+0000","last_became_peered":"2024-09-19T04:04:46.362529+0000","last_unstale":"2024-09-19T04:05:42.332887+0000","last_undegraded":"2024-09-19T04:05:42.332887+0000","last_fullsized":"2024-09-19T04:05:42.332887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:01.821697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331910+0000","last_change":"2024-09-19T04:04:39.323499+0000","last_active":"2024-09-19T04:05:42.331910+0000","last_peered":"2024-09-19T04:05:42.331910+0000","last_clean":"2024-09-19T04:05:42.331910+0000","last_became_active":"2024-09-19T04:04:39.323076+0000","last_became_peered":"2024-09-19T04:04:39.323076+0000","last_unstale":"2024-09-19T04:05:42.331910+0000","last_undegraded":"2024-09-19T04:05:42.331910+0000","last_fullsized":"2024-09-19T04:05:42.331910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:10.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336750+0000","last_change":"2024-09-19T04:04:37.647927+0000","last_active":"2024-09-19T04:05:42.336750+0000","last_peered":"2024-09-19T04:05:42.336750+0000","last_clean":"2024-09-19T04:05:42.336750+0000","last_became_active":"2024-09-19T04:04:37.647279+0000","last_became_peered":"2024-09-19T04:04:37.647279+0000","last_unstale":"2024-09-19T04:05:42.336750+0000","last_undegraded":"2024-09-19T04:05:42.336750+0000","last_fullsized":"2024-09-19T04:05:42.336750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:05:50.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337020+0000","last_change":"2024-09-19T04:04:46.363562+0000","last_active":"2024-09-19T04:05:42.337020+0000","last_peered":"2024-09-19T04:05:42.337020+0000","last_clean":"2024-09-19T04:05:42.337020+0000","last_became_active":"2024-09-19T04:04:46.362915+0000","last_became_peered":"2024-09-19T04:04:46.362915+0000","last_unstale":"2024-09-19T04:05:42.337020+0000","last_undegraded":"2024-09-19T04:05:42.337020+0000","last_fullsized":"2024-09-19T04:05:42.337020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:08.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150482+0000","last_change":"2024-09-19T04:04:39.329034+0000","last_active":"2024-09-19T04:05:43.150482+0000","last_peered":"2024-09-19T04:05:43.150482+0000","last_clean":"2024-09-19T04:05:43.150482+0000","last_became_active":"2024-09-19T04:04:39.328692+0000","last_became_peered":"2024-09-19T04:04:39.328692+0000","last_unstale":"2024-09-19T04:05:43.150482+0000","last_undegraded":"2024-09-19T04:05:43.150482+0000","last_fullsized":"2024-09-19T04:05:43.150482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:42.903089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338635+0000","last_change":"2024-09-19T04:04:37.212393+0000","last_active":"2024-09-19T04:05:42.338635+0000","last_peered":"2024-09-19T04:05:42.338635+0000","last_clean":"2024-09-19T04:05:42.338635+0000","last_became_active":"2024-09-19T04:04:37.212013+0000","last_became_peered":"2024-09-19T04:04:37.212013+0000","last_unstale":"2024-09-19T04:05:42.338635+0000","last_undegraded":"2024-09-19T04:05:42.338635+0000","last_fullsized":"2024-09-19T04:05:42.338635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:37.981075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.151378+0000","last_change":"2024-09-19T04:05:43.151378+0000","last_active":"2024-09-19T04:05:43.151378+0000","last_peered":"2024-09-19T04:05:43.151378+0000","last_clean":"2024-09-19T04:05:43.151378+0000","last_became_active":"2024-09-19T04:05:43.150907+0000","last_became_peered":"2024-09-19T04:05:43.150907+0000","last_unstale":"2024-09-19T04:05:43.151378+0000","last_undegraded":"2024-09-19T04:05:43.151378+0000","last_fullsized":"2024-09-19T04:05:43.151378+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:09.542371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332236+0000","last_change":"2024-09-19T04:04:39.319705+0000","last_active":"2024-09-19T04:05:42.332236+0000","last_peered":"2024-09-19T04:05:42.332236+0000","last_clean":"2024-09-19T04:05:42.332236+0000","last_became_active":"2024-09-19T04:04:39.318065+0000","last_became_peered":"2024-09-19T04:04:39.318065+0000","last_unstale":"2024-09-19T04:05:42.332236+0000","last_undegraded":"2024-09-19T04:05:42.332236+0000","last_fullsized":"2024-09-19T04:05:42.332236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:15.630927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:12.677758+0000","last_change":"2024-09-19T04:04:37.215192+0000","last_active":"2024-09-19T04:06:12.677758+0000","last_peered":"2024-09-19T04:06:12.677758+0000","last_clean":"2024-09-19T04:06:12.677758+0000","last_became_active":"2024-09-19T04:04:37.214872+0000","last_became_peered":"2024-09-19T04:04:37.214872+0000","last_unstale":"2024-09-19T04:06:12.677758+0000","last_undegraded":"2024-09-19T04:06:12.677758+0000","last_fullsized":"2024-09-19T04:06:12.677758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:18.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822243+0000","last_change":"2024-09-19T04:04:46.358113+0000","last_active":"2024-09-19T04:05:42.822243+0000","last_peered":"2024-09-19T04:05:42.822243+0000","last_clean":"2024-09-19T04:05:42.822243+0000","last_became_active":"2024-09-19T04:04:46.356604+0000","last_became_peered":"2024-09-19T04:04:46.356604+0000","last_unstale":"2024-09-19T04:05:42.822243+0000","last_undegraded":"2024-09-19T04:05:42.822243+0000","last_fullsized":"2024-09-19T04:05:42.822243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:38.147738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.149511+0000","last_change":"2024-09-19T04:04:46.354516+0000","last_active":"2024-09-19T04:05:43.149511+0000","last_peered":"2024-09-19T04:05:43.149511+0000","last_clean":"2024-09-19T04:05:43.149511+0000","last_became_active":"2024-09-19T04:04:46.354034+0000","last_became_peered":"2024-09-19T04:04:46.354034+0000","last_unstale":"2024-09-19T04:05:43.149511+0000","last_undegraded":"2024-09-19T04:05:43.149511+0000","last_fullsized":"2024-09-19T04:05:43.149511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:21.946746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332777+0000","last_change":"2024-09-19T04:04:39.311324+0000","last_active":"2024-09-19T04:05:42.332777+0000","last_peered":"2024-09-19T04:05:42.332777+0000","last_clean":"2024-09-19T04:05:42.332777+0000","last_became_active":"2024-09-19T04:04:39.310977+0000","last_became_peered":"2024-09-19T04:04:39.310977+0000","last_unstale":"2024-09-19T04:05:42.332777+0000","last_undegraded":"2024-09-19T04:05:42.332777+0000","last_fullsized":"2024-09-19T04:05:42.332777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:43.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335659+0000","last_change":"2024-09-19T04:04:37.212773+0000","last_active":"2024-09-19T04:05:42.335659+0000","last_peered":"2024-09-19T04:05:42.335659+0000","last_clean":"2024-09-19T04:05:42.335659+0000","last_became_active":"2024-09-19T04:04:37.212405+0000","last_became_peered":"2024-09-19T04:04:37.212405+0000","last_unstale":"2024-09-19T04:05:42.335659+0000","last_undegraded":"2024-09-19T04:05:42.335659+0000","last_fullsized":"2024-09-19T04:05:42.335659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:29.020122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334347+0000","last_change":"2024-09-19T04:04:46.363052+0000","last_active":"2024-09-19T04:05:42.334347+0000","last_peered":"2024-09-19T04:05:42.334347+0000","last_clean":"2024-09-19T04:05:42.334347+0000","last_became_active":"2024-09-19T04:04:46.362588+0000","last_became_peered":"2024-09-19T04:04:46.362588+0000","last_unstale":"2024-09-19T04:05:42.334347+0000","last_undegraded":"2024-09-19T04:05:42.334347+0000","last_fullsized":"2024-09-19T04:05:42.334347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:52.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822622+0000","last_change":"2024-09-19T04:04:39.313686+0000","last_active":"2024-09-19T04:05:42.822622+0000","last_peered":"2024-09-19T04:05:42.822622+0000","last_clean":"2024-09-19T04:05:42.822622+0000","last_became_active":"2024-09-19T04:04:39.313386+0000","last_became_peered":"2024-09-19T04:04:39.313386+0000","last_unstale":"2024-09-19T04:05:42.822622+0000","last_undegraded":"2024-09-19T04:05:42.822622+0000","last_fullsized":"2024-09-19T04:05:42.822622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:08.274930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464556+0000","last_change":"2024-09-19T04:04:37.207926+0000","last_active":"2024-09-19T04:05:42.464556+0000","last_peered":"2024-09-19T04:05:42.464556+0000","last_clean":"2024-09-19T04:05:42.464556+0000","last_became_active":"2024-09-19T04:04:37.207538+0000","last_became_peered":"2024-09-19T04:04:37.207538+0000","last_unstale":"2024-09-19T04:05:42.464556+0000","last_undegraded":"2024-09-19T04:05:42.464556+0000","last_fullsized":"2024-09-19T04:05:42.464556+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:29.621638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334223+0000","last_change":"2024-09-19T04:04:37.212829+0000","last_active":"2024-09-19T04:05:42.334223+0000","last_peered":"2024-09-19T04:05:42.334223+0000","last_clean":"2024-09-19T04:05:42.334223+0000","last_became_active":"2024-09-19T04:04:37.212294+0000","last_became_peered":"2024-09-19T04:04:37.212294+0000","last_unstale":"2024-09-19T04:05:42.334223+0000","last_undegraded":"2024-09-19T04:05:42.334223+0000","last_fullsized":"2024-09-19T04:05:42.334223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:34.479708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335826+0000","last_change":"2024-09-19T04:04:39.328622+0000","last_active":"2024-09-19T04:05:42.335826+0000","last_peered":"2024-09-19T04:05:42.335826+0000","last_clean":"2024-09-19T04:05:42.335826+0000","last_became_active":"2024-09-19T04:04:39.328290+0000","last_became_peered":"2024-09-19T04:04:39.328290+0000","last_unstale":"2024-09-19T04:05:42.335826+0000","last_undegraded":"2024-09-19T04:05:42.335826+0000","last_fullsized":"2024-09-19T04:05:42.335826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:04.290767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331562+0000","last_change":"2024-09-19T04:04:46.362853+0000","last_active":"2024-09-19T04:05:42.331562+0000","last_peered":"2024-09-19T04:05:42.331562+0000","last_clean":"2024-09-19T04:05:42.331562+0000","last_became_active":"2024-09-19T04:04:46.362392+0000","last_became_peered":"2024-09-19T04:04:46.362392+0000","last_unstale":"2024-09-19T04:05:42.331562+0000","last_undegraded":"2024-09-19T04:05:42.331562+0000","last_fullsized":"2024-09-19T04:05:42.331562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:40.016502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:13.680140+0000","last_change":"2024-09-19T04:04:37.213248+0000","last_active":"2024-09-19T04:06:13.680140+0000","last_peered":"2024-09-19T04:06:13.680140+0000","last_clean":"2024-09-19T04:06:13.680140+0000","last_became_active":"2024-09-19T04:04:37.212639+0000","last_became_peered":"2024-09-19T04:04:37.212639+0000","last_unstale":"2024-09-19T04:06:13.680140+0000","last_undegraded":"2024-09-19T04:06:13.680140+0000","last_fullsized":"2024-09-19T04:06:13.680140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:45:05.917792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333052+0000","last_change":"2024-09-19T04:04:39.330160+0000","last_active":"2024-09-19T04:05:42.333052+0000","last_peered":"2024-09-19T04:05:42.333052+0000","last_clean":"2024-09-19T04:05:42.333052+0000","last_became_active":"2024-09-19T04:04:39.329828+0000","last_became_peered":"2024-09-19T04:04:39.329828+0000","last_unstale":"2024-09-19T04:05:42.333052+0000","last_undegraded":"2024-09-19T04:05:42.333052+0000","last_fullsized":"2024-09-19T04:05:42.333052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:57.712550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332247+0000","last_change":"2024-09-19T04:04:46.360459+0000","last_active":"2024-09-19T04:05:42.332247+0000","last_peered":"2024-09-19T04:05:42.332247+0000","last_clean":"2024-09-19T04:05:42.332247+0000","last_became_active":"2024-09-19T04:04:46.359914+0000","last_became_peered":"2024-09-19T04:04:46.359914+0000","last_unstale":"2024-09-19T04:05:42.332247+0000","last_undegraded":"2024-09-19T04:05:42.332247+0000","last_fullsized":"2024-09-19T04:05:42.332247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:30.380358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333565+0000","last_change":"2024-09-19T04:04:39.324847+0000","last_active":"2024-09-19T04:05:42.333565+0000","last_peered":"2024-09-19T04:05:42.333565+0000","last_clean":"2024-09-19T04:05:42.333565+0000","last_became_active":"2024-09-19T04:04:39.324402+0000","last_became_peered":"2024-09-19T04:04:39.324402+0000","last_unstale":"2024-09-19T04:05:42.333565+0000","last_undegraded":"2024-09-19T04:05:42.333565+0000","last_fullsized":"2024-09-19T04:05:42.333565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:41.996135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.339436+0000","last_change":"2024-09-19T04:05:42.339436+0000","last_active":"2024-09-19T04:05:42.339436+0000","last_peered":"2024-09-19T04:05:42.339436+0000","last_clean":"2024-09-19T04:05:42.339436+0000","last_became_active":"2024-09-19T04:05:42.339048+0000","last_became_peered":"2024-09-19T04:05:42.339048+0000","last_unstale":"2024-09-19T04:05:42.339436+0000","last_undegraded":"2024-09-19T04:05:42.339436+0000","last_fullsized":"2024-09-19T04:05:42.339436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:53.175494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331023+0000","last_change":"2024-09-19T04:04:39.318556+0000","last_active":"2024-09-19T04:05:42.331023+0000","last_peered":"2024-09-19T04:05:42.331023+0000","last_clean":"2024-09-19T04:05:42.331023+0000","last_became_active":"2024-09-19T04:04:39.318200+0000","last_became_peered":"2024-09-19T04:04:39.318200+0000","last_unstale":"2024-09-19T04:05:42.331023+0000","last_undegraded":"2024-09-19T04:05:42.331023+0000","last_fullsized":"2024-09-19T04:05:42.331023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:40.847440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150170+0000","last_change":"2024-09-19T04:04:39.320698+0000","last_active":"2024-09-19T04:05:43.150170+0000","last_peered":"2024-09-19T04:05:43.150170+0000","last_clean":"2024-09-19T04:05:43.150170+0000","last_became_active":"2024-09-19T04:04:39.320419+0000","last_became_peered":"2024-09-19T04:04:39.320419+0000","last_unstale":"2024-09-19T04:05:43.150170+0000","last_undegraded":"2024-09-19T04:05:43.150170+0000","last_fullsized":"2024-09-19T04:05:43.150170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:32.708394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464888+0000","last_change":"2024-09-19T04:04:39.319886+0000","last_active":"2024-09-19T04:05:42.464888+0000","last_peered":"2024-09-19T04:05:42.464888+0000","last_clean":"2024-09-19T04:05:42.464888+0000","last_became_active":"2024-09-19T04:04:39.319217+0000","last_became_peered":"2024-09-19T04:04:39.319217+0000","last_unstale":"2024-09-19T04:05:42.464888+0000","last_undegraded":"2024-09-19T04:05:42.464888+0000","last_fullsized":"2024-09-19T04:05:42.464888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:38.829608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334219+0000","last_change":"2024-09-19T04:04:39.327639+0000","last_active":"2024-09-19T04:05:42.334219+0000","last_peered":"2024-09-19T04:05:42.334219+0000","last_clean":"2024-09-19T04:05:42.334219+0000","last_became_active":"2024-09-19T04:04:39.327349+0000","last_became_peered":"2024-09-19T04:04:39.327349+0000","last_unstale":"2024-09-19T04:05:42.334219+0000","last_undegraded":"2024-09-19T04:05:42.334219+0000","last_fullsized":"2024-09-19T04:05:42.334219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:31.056609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332017+0000","last_change":"2024-09-19T04:04:39.320253+0000","last_active":"2024-09-19T04:05:42.332017+0000","last_peered":"2024-09-19T04:05:42.332017+0000","last_clean":"2024-09-19T04:05:42.332017+0000","last_became_active":"2024-09-19T04:04:39.319851+0000","last_became_peered":"2024-09-19T04:04:39.319851+0000","last_unstale":"2024-09-19T04:05:42.332017+0000","last_undegraded":"2024-09-19T04:05:42.332017+0000","last_fullsized":"2024-09-19T04:05:42.332017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:53.631216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.823059+0000","last_change":"2024-09-19T04:04:39.322779+0000","last_active":"2024-09-19T04:05:42.823059+0000","last_peered":"2024-09-19T04:05:42.823059+0000","last_clean":"2024-09-19T04:05:42.823059+0000","last_became_active":"2024-09-19T04:04:39.322498+0000","last_became_peered":"2024-09-19T04:04:39.322498+0000","last_unstale":"2024-09-19T04:05:42.823059+0000","last_undegraded":"2024-09-19T04:05:42.823059+0000","last_fullsized":"2024-09-19T04:05:42.823059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:24:11.956002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335250+0000","last_change":"2024-09-19T04:04:39.322759+0000","last_active":"2024-09-19T04:05:42.335250+0000","last_peered":"2024-09-19T04:05:42.335250+0000","last_clean":"2024-09-19T04:05:42.335250+0000","last_became_active":"2024-09-19T04:04:39.322429+0000","last_became_peered":"2024-09-19T04:04:39.322429+0000","last_unstale":"2024-09-19T04:05:42.335250+0000","last_undegraded":"2024-09-19T04:05:42.335250+0000","last_fullsized":"2024-09-19T04:05:42.335250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:36.069931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822800+0000","last_change":"2024-09-19T04:04:39.320078+0000","last_active":"2024-09-19T04:05:42.822800+0000","last_peered":"2024-09-19T04:05:42.822800+0000","last_clean":"2024-09-19T04:05:42.822800+0000","last_became_active":"2024-09-19T04:04:39.319755+0000","last_became_peered":"2024-09-19T04:04:39.319755+0000","last_unstale":"2024-09-19T04:05:42.822800+0000","last_undegraded":"2024-09-19T04:05:42.822800+0000","last_fullsized":"2024-09-19T04:05:42.822800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:07.651712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331888+0000","last_change":"2024-09-19T04:04:39.319839+0000","last_active":"2024-09-19T04:05:42.331888+0000","last_peered":"2024-09-19T04:05:42.331888+0000","last_clean":"2024-09-19T04:05:42.331888+0000","last_became_active":"2024-09-19T04:04:39.319527+0000","last_became_peered":"2024-09-19T04:04:39.319527+0000","last_unstale":"2024-09-19T04:05:42.331888+0000","last_undegraded":"2024-09-19T04:05:42.331888+0000","last_fullsized":"2024-09-19T04:05:42.331888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:31.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.465177+0000","last_change":"2024-09-19T04:04:39.315975+0000","last_active":"2024-09-19T04:05:42.465177+0000","last_peered":"2024-09-19T04:05:42.465177+0000","last_clean":"2024-09-19T04:05:42.465177+0000","last_became_active":"2024-09-19T04:04:39.315671+0000","last_became_peered":"2024-09-19T04:04:39.315671+0000","last_unstale":"2024-09-19T04:05:42.465177+0000","last_undegraded":"2024-09-19T04:05:42.465177+0000","last_fullsized":"2024-09-19T04:05:42.465177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:16.349853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333449+0000","last_change":"2024-09-19T04:04:39.316278+0000","last_active":"2024-09-19T04:05:42.333449+0000","last_peered":"2024-09-19T04:05:42.333449+0000","last_clean":"2024-09-19T04:05:42.333449+0000","last_became_active":"2024-09-19T04:04:39.315876+0000","last_became_peered":"2024-09-19T04:04:39.315876+0000","last_unstale":"2024-09-19T04:05:42.333449+0000","last_undegraded":"2024-09-19T04:05:42.333449+0000","last_fullsized":"2024-09-19T04:05:42.333449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:23.544405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332393+0000","last_change":"2024-09-19T04:04:39.318952+0000","last_active":"2024-09-19T04:05:42.332393+0000","last_peered":"2024-09-19T04:05:42.332393+0000","last_clean":"2024-09-19T04:05:42.332393+0000","last_became_active":"2024-09-19T04:04:39.318598+0000","last_became_peered":"2024-09-19T04:04:39.318598+0000","last_unstale":"2024-09-19T04:05:42.332393+0000","last_undegraded":"2024-09-19T04:05:42.332393+0000","last_fullsized":"2024-09-19T04:05:42.332393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:12.654611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333268+0000","last_change":"2024-09-19T04:04:39.311334+0000","last_active":"2024-09-19T04:05:42.333268+0000","last_peered":"2024-09-19T04:05:42.333268+0000","last_clean":"2024-09-19T04:05:42.333268+0000","last_became_active":"2024-09-19T04:04:39.311038+0000","last_became_peered":"2024-09-19T04:04:39.311038+0000","last_unstale":"2024-09-19T04:05:42.333268+0000","last_undegraded":"2024-09-19T04:05:42.333268+0000","last_fullsized":"2024-09-19T04:05:42.333268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:41.655363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336230+0000","last_change":"2024-09-19T04:04:39.319529+0000","last_active":"2024-09-19T04:05:42.336230+0000","last_peered":"2024-09-19T04:05:42.336230+0000","last_clean":"2024-09-19T04:05:42.336230+0000","last_became_active":"2024-09-19T04:04:39.318787+0000","last_became_peered":"2024-09-19T04:04:39.318787+0000","last_unstale":"2024-09-19T04:05:42.336230+0000","last_undegraded":"2024-09-19T04:05:42.336230+0000","last_fullsized":"2024-09-19T04:05:42.336230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.209422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331537+0000","last_change":"2024-09-19T04:04:39.320320+0000","last_active":"2024-09-19T04:05:42.331537+0000","last_peered":"2024-09-19T04:05:42.331537+0000","last_clean":"2024-09-19T04:05:42.331537+0000","last_became_active":"2024-09-19T04:04:39.319091+0000","last_became_peered":"2024-09-19T04:04:39.319091+0000","last_unstale":"2024-09-19T04:05:42.331537+0000","last_undegraded":"2024-09-19T04:05:42.331537+0000","last_fullsized":"2024-09-19T04:05:42.331537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:39.495079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.149666+0000","last_change":"2024-09-19T04:04:39.323270+0000","last_active":"2024-09-19T04:05:43.149666+0000","last_peered":"2024-09-19T04:05:43.149666+0000","last_clean":"2024-09-19T04:05:43.149666+0000","last_became_active":"2024-09-19T04:04:39.322890+0000","last_became_peered":"2024-09-19T04:04:39.322890+0000","last_unstale":"2024-09-19T04:05:43.149666+0000","last_undegraded":"2024-09-19T04:05:43.149666+0000","last_fullsized":"2024-09-19T04:05:43.149666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:07.311654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150098+0000","last_change":"2024-09-19T04:04:39.317098+0000","last_active":"2024-09-19T04:05:43.150098+0000","last_peered":"2024-09-19T04:05:43.150098+0000","last_clean":"2024-09-19T04:05:43.150098+0000","last_became_active":"2024-09-19T04:04:39.316792+0000","last_became_peered":"2024-09-19T04:04:39.316792+0000","last_unstale":"2024-09-19T04:05:43.150098+0000","last_undegraded":"2024-09-19T04:05:43.150098+0000","last_fullsized":"2024-09-19T04:05:43.150098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.571149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.465176+0000","last_change":"2024-09-19T04:04:39.326473+0000","last_active":"2024-09-19T04:05:42.465176+0000","last_peered":"2024-09-19T04:05:42.465176+0000","last_clean":"2024-09-19T04:05:42.465176+0000","last_became_active":"2024-09-19T04:04:39.325506+0000","last_became_peered":"2024-09-19T04:04:39.325506+0000","last_unstale":"2024-09-19T04:05:42.465176+0000","last_undegraded":"2024-09-19T04:05:42.465176+0000","last_fullsized":"2024-09-19T04:05:42.465176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:40.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337032+0000","last_change":"2024-09-19T04:04:39.323536+0000","last_active":"2024-09-19T04:05:42.337032+0000","last_peered":"2024-09-19T04:05:42.337032+0000","last_clean":"2024-09-19T04:05:42.337032+0000","last_became_active":"2024-09-19T04:04:39.323224+0000","last_became_peered":"2024-09-19T04:04:39.323224+0000","last_unstale":"2024-09-19T04:05:42.337032+0000","last_undegraded":"2024-09-19T04:05:42.337032+0000","last_fullsized":"2024-09-19T04:05:42.337032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:54.640599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338305+0000","last_change":"2024-09-19T04:04:39.323802+0000","last_active":"2024-09-19T04:05:42.338305+0000","last_peered":"2024-09-19T04:05:42.338305+0000","last_clean":"2024-09-19T04:05:42.338305+0000","last_became_active":"2024-09-19T04:04:39.323031+0000","last_became_peered":"2024-09-19T04:04:39.323031+0000","last_unstale":"2024-09-19T04:05:42.338305+0000","last_undegraded":"2024-09-19T04:05:42.338305+0000","last_fullsized":"2024-09-19T04:05:42.338305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:12.467182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334632+0000","last_change":"2024-09-19T04:04:39.324870+0000","last_active":"2024-09-19T04:05:42.334632+0000","last_peered":"2024-09-19T04:05:42.334632+0000","last_clean":"2024-09-19T04:05:42.334632+0000","last_became_active":"2024-09-19T04:04:39.324551+0000","last_became_peered":"2024-09-19T04:04:39.324551+0000","last_unstale":"2024-09-19T04:05:42.334632+0000","last_undegraded":"2024-09-19T04:05:42.334632+0000","last_fullsized":"2024-09-19T04:05:42.334632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:34.766437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332266+0000","last_change":"2024-09-19T04:04:39.329086+0000","last_active":"2024-09-19T04:05:42.332266+0000","last_peered":"2024-09-19T04:05:42.332266+0000","last_clean":"2024-09-19T04:05:42.332266+0000","last_became_active":"2024-09-19T04:04:39.328651+0000","last_became_peered":"2024-09-19T04:04:39.328651+0000","last_unstale":"2024-09-19T04:05:42.332266+0000","last_undegraded":"2024-09-19T04:05:42.332266+0000","last_fullsized":"2024-09-19T04:05:42.332266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:12.079162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822563+0000","last_change":"2024-09-19T04:04:39.325427+0000","last_active":"2024-09-19T04:05:42.822563+0000","last_peered":"2024-09-19T04:05:42.822563+0000","last_clean":"2024-09-19T04:05:42.822563+0000","last_became_active":"2024-09-19T04:04:39.325113+0000","last_became_peered":"2024-09-19T04:04:39.325113+0000","last_unstale":"2024-09-19T04:05:42.822563+0000","last_undegraded":"2024-09-19T04:05:42.822563+0000","last_fullsized":"2024-09-19T04:05:42.822563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:15.844503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337085+0000","last_change":"2024-09-19T04:04:39.325032+0000","last_active":"2024-09-19T04:05:42.337085+0000","last_peered":"2024-09-19T04:05:42.337085+0000","last_clean":"2024-09-19T04:05:42.337085+0000","last_became_active":"2024-09-19T04:04:39.324618+0000","last_became_peered":"2024-09-19T04:04:39.324618+0000","last_unstale":"2024-09-19T04:05:42.337085+0000","last_undegraded":"2024-09-19T04:05:42.337085+0000","last_fullsized":"2024-09-19T04:05:42.337085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:58.748613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334110+0000","last_change":"2024-09-19T04:04:39.318336+0000","last_active":"2024-09-19T04:05:42.334110+0000","last_peered":"2024-09-19T04:05:42.334110+0000","last_clean":"2024-09-19T04:05:42.334110+0000","last_became_active":"2024-09-19T04:04:39.317032+0000","last_became_peered":"2024-09-19T04:04:39.317032+0000","last_unstale":"2024-09-19T04:05:42.334110+0000","last_undegraded":"2024-09-19T04:05:42.334110+0000","last_fullsized":"2024-09-19T04:05:42.334110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:04.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332708+0000","last_change":"2024-09-19T04:04:39.321909+0000","last_active":"2024-09-19T04:05:42.332708+0000","last_peered":"2024-09-19T04:05:42.332708+0000","last_clean":"2024-09-19T04:05:42.332708+0000","last_became_active":"2024-09-19T04:04:39.320522+0000","last_became_peered":"2024-09-19T04:04:39.320522+0000","last_unstale":"2024-09-19T04:05:42.332708+0000","last_undegraded":"2024-09-19T04:05:42.332708+0000","last_fullsized":"2024-09-19T04:05:42.332708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:57.270998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336942+0000","last_change":"2024-09-19T04:04:39.324610+0000","last_active":"2024-09-19T04:05:42.336942+0000","last_peered":"2024-09-19T04:05:42.336942+0000","last_clean":"2024-09-19T04:05:42.336942+0000","last_became_active":"2024-09-19T04:04:39.324190+0000","last_became_peered":"2024-09-19T04:04:39.324190+0000","last_unstale":"2024-09-19T04:05:42.336942+0000","last_undegraded":"2024-09-19T04:05:42.336942+0000","last_fullsized":"2024-09-19T04:05:42.336942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:51.418210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.149987+0000","last_change":"2024-09-19T04:04:39.324220+0000","last_active":"2024-09-19T04:05:43.149987+0000","last_peered":"2024-09-19T04:05:43.149987+0000","last_clean":"2024-09-19T04:05:43.149987+0000","last_became_active":"2024-09-19T04:04:39.323890+0000","last_became_peered":"2024-09-19T04:04:39.323890+0000","last_unstale":"2024-09-19T04:05:43.149987+0000","last_undegraded":"2024-09-19T04:05:43.149987+0000","last_fullsized":"2024-09-19T04:05:43.149987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:15.475088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150226+0000","last_change":"2024-09-19T04:04:39.317240+0000","last_active":"2024-09-19T04:05:43.150226+0000","last_peered":"2024-09-19T04:05:43.150226+0000","last_clean":"2024-09-19T04:05:43.150226+0000","last_became_active":"2024-09-19T04:04:39.316740+0000","last_became_peered":"2024-09-19T04:04:39.316740+0000","last_unstale":"2024-09-19T04:05:43.150226+0000","last_undegraded":"2024-09-19T04:05:43.150226+0000","last_fullsized":"2024-09-19T04:05:43.150226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:46.705467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336150+0000","last_change":"2024-09-19T04:04:39.319665+0000","last_active":"2024-09-19T04:05:42.336150+0000","last_peered":"2024-09-19T04:05:42.336150+0000","last_clean":"2024-09-19T04:05:42.336150+0000","last_became_active":"2024-09-19T04:04:39.319227+0000","last_became_peered":"2024-09-19T04:04:39.319227+0000","last_unstale":"2024-09-19T04:05:42.336150+0000","last_undegraded":"2024-09-19T04:05:42.336150+0000","last_fullsized":"2024-09-19T04:05:42.336150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:45.581254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68891,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":553,"num_read_kb":351,"num_write":246,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":60960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":206673,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248101,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481825,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":510207,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710757,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1159077,"data_compressed":59360,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874283,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":514211,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":510471,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364853,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1081344,"data_stored":1106992,"data_compressed":59360,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528379,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":506012,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1089536,"data_stored":1102181,"data_compressed":59360,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":571769,"data_compressed":33234,"data_compressed_allocated":45056,"data_compressed_original":122880,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":505327,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":506869,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":569118,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":507075,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":62776,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:06:17.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:06:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:17.876 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:17 smithi019 ceph-mon[30938]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:17.877 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:17 smithi019 ceph-mon[30938]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:06:17.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:17 smithi111 ceph-mon[35365]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:17.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:17 smithi111 ceph-mon[35365]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:06:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:17 smithi196 ceph-mon[35731]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:17.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:17 smithi196 ceph-mon[35731]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:06:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 -- 172.21.15.19:0/2760907015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518101de0 msgr2=0x7f15181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2760907015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518101de0 0x7f15181021c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1508009900 tx=0x7f150802a410 comp rx=0 tx=0).stop 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 -- 172.21.15.19:0/2760907015 shutdown_connections 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2760907015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518106f10 0x7f151810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2760907015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1518102790 0x7f15181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2760907015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518101de0 0x7f15181021c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 -- 172.21.15.19:0/2760907015 >> 172.21.15.19:0/2760907015 conn(0x7f15180fd8c0 msgr2=0x7f15180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 -- 172.21.15.19:0/2760907015 shutdown_connections 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.957+0000 7f151cdc0640 1 -- 172.21.15.19:0/2760907015 wait complete. 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151cdc0640 1 Processor -- start 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151cdc0640 1 -- start start 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151cdc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518101de0 0x7f151819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151cdc0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518102790 0x7f151819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151cdc0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1518106f10 0x7f151819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151cdc0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1518111f90 con 0x7f1518102790 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151cdc0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1518111e10 con 0x7f1518106f10 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151cdc0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1518112110 con 0x7f1518101de0 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f1515d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518102790 0x7f151819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f1515d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518102790 0x7f151819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38430/0 (socket says 172.21.15.19:38430) 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f1515d9b640 1 -- 172.21.15.19:0/2905086835 learned_addr learned my addr 172.21.15.19:0/2905086835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f151659c640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518101de0 0x7f151819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.958+0000 7f1516d9d640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1518106f10 0x7f151819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f1515d9b640 1 -- 172.21.15.19:0/2905086835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518101de0 msgr2=0x7f151819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f1515d9b640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518101de0 0x7f151819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f1515d9b640 1 -- 172.21.15.19:0/2905086835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1518106f10 msgr2=0x7f151819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f1515d9b640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1518106f10 0x7f151819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f1515d9b640 1 -- 172.21.15.19:0/2905086835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151819f6a0 con 0x7f1518102790 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f151659c640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518101de0 0x7f151819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f1516d9d640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1518106f10 0x7f151819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f1515d9b640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518102790 0x7f151819ac30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f150000b4a0 tx=0x7f150000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f150f7fe640 1 -- 172.21.15.19:0/2905086835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500014070 con 0x7f1518102790 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151819f990 con 0x7f1518102790 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f150f7fe640 1 -- 172.21.15.19:0/2905086835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1500010af0 con 0x7f1518102790 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f150f7fe640 1 -- 172.21.15.19:0/2905086835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500017d10 con 0x7f1518102790 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.959+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15181a7270 con 0x7f1518102790 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.960+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15181021c0 con 0x7f1518102790 2024-09-19T04:06:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.961+0000 7f150f7fe640 1 -- 172.21.15.19:0/2905086835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1500017070 con 0x7f1518102790 2024-09-19T04:06:17.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.962+0000 7f150f7fe640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15040a32d0 0x7f15040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:17.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.962+0000 7f150f7fe640 1 -- 172.21.15.19:0/2905086835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f15000cf150 con 0x7f1518102790 2024-09-19T04:06:17.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.962+0000 7f151659c640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15040a32d0 0x7f15040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:17.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.962+0000 7f151659c640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15040a32d0 0x7f15040a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1508000c00 tx=0x7f1508032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:17.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:17.966+0000 7f150f7fe640 1 -- 172.21.15.19:0/2905086835 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f150009bae0 con 0x7f1518102790 2024-09-19T04:06:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.139+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1518075430 con 0x7f15040a32d0 2024-09-19T04:06:18.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.147+0000 7f150f7fe640 1 -- 172.21.15.19:0/2905086835 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340690 (secure 0 0 0) 0x7f1518075430 con 0x7f15040a32d0 2024-09-19T04:06:18.151 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:18.154 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-19T04:06:18.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15040a32d0 msgr2=0x7f15040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15040a32d0 0x7f15040a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1508000c00 tx=0x7f1508032d50 comp rx=0 tx=0).stop 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518102790 msgr2=0x7f151819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518102790 0x7f151819ac30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f150000b4a0 tx=0x7f150000b960 comp rx=0 tx=0).stop 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 shutdown_connections 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f15040a32d0 0x7f15040a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1518106f10 0x7f151819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1518102790 0x7f151819ac30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 --2- 172.21.15.19:0/2905086835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518101de0 0x7f151819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 >> 172.21.15.19:0/2905086835 conn(0x7f15180fd8c0 msgr2=0x7f15180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 shutdown_connections 2024-09-19T04:06:18.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:18.150+0000 7f151cdc0640 1 -- 172.21.15.19:0/2905086835 wait complete. 2024-09-19T04:06:18.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:18 smithi019 ceph-mon[30938]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:06:18.845 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-19T04:06:16.433017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659493,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":679,"num_read_kb":460,"num_write":429,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1241696,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349104,"kb_used_data":12300,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592720,"statfs":{"total":1151940427776,"available":1151582945280,"internally_reserved":0,"allocated":12595200,"data_stored":8069309,"data_compressed":398906,"data_compressed_allocated":540672,"data_compressed_original":2613248,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003100"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:11.672149+0000","last_change":"2024-09-19T04:04:37.214769+0000","last_active":"2024-09-19T04:06:11.672149+0000","last_peered":"2024-09-19T04:06:11.672149+0000","last_clean":"2024-09-19T04:06:11.672149+0000","last_became_active":"2024-09-19T04:04:37.214518+0000","last_became_peered":"2024-09-19T04:04:37.214518+0000","last_unstale":"2024-09-19T04:06:11.672149+0000","last_undegraded":"2024-09-19T04:06:11.672149+0000","last_fullsized":"2024-09-19T04:06:11.672149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:50.704566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822826+0000","last_change":"2024-09-19T04:04:39.329733+0000","last_active":"2024-09-19T04:05:42.822826+0000","last_peered":"2024-09-19T04:05:42.822826+0000","last_clean":"2024-09-19T04:05:42.822826+0000","last_became_active":"2024-09-19T04:04:39.329031+0000","last_became_peered":"2024-09-19T04:04:39.329031+0000","last_unstale":"2024-09-19T04:05:42.822826+0000","last_undegraded":"2024-09-19T04:05:42.822826+0000","last_fullsized":"2024-09-19T04:05:42.822826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:57.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.330991+0000","last_change":"2024-09-19T04:04:46.362679+0000","last_active":"2024-09-19T04:05:42.330991+0000","last_peered":"2024-09-19T04:05:42.330991+0000","last_clean":"2024-09-19T04:05:42.330991+0000","last_became_active":"2024-09-19T04:04:46.362257+0000","last_became_peered":"2024-09-19T04:04:46.362257+0000","last_unstale":"2024-09-19T04:05:42.330991+0000","last_undegraded":"2024-09-19T04:05:42.330991+0000","last_fullsized":"2024-09-19T04:05:42.330991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.389839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.823042+0000","last_change":"2024-09-19T04:04:37.647295+0000","last_active":"2024-09-19T04:05:42.823042+0000","last_peered":"2024-09-19T04:05:42.823042+0000","last_clean":"2024-09-19T04:05:42.823042+0000","last_became_active":"2024-09-19T04:04:37.646964+0000","last_became_peered":"2024-09-19T04:04:37.646964+0000","last_unstale":"2024-09-19T04:05:42.823042+0000","last_undegraded":"2024-09-19T04:05:42.823042+0000","last_fullsized":"2024-09-19T04:05:42.823042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:49.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150451+0000","last_change":"2024-09-19T04:04:39.318270+0000","last_active":"2024-09-19T04:05:43.150451+0000","last_peered":"2024-09-19T04:05:43.150451+0000","last_clean":"2024-09-19T04:05:43.150451+0000","last_became_active":"2024-09-19T04:04:39.317972+0000","last_became_peered":"2024-09-19T04:04:39.317972+0000","last_unstale":"2024-09-19T04:05:43.150451+0000","last_undegraded":"2024-09-19T04:05:43.150451+0000","last_fullsized":"2024-09-19T04:05:43.150451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:25.994112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336125+0000","last_change":"2024-09-19T04:04:46.355111+0000","last_active":"2024-09-19T04:05:42.336125+0000","last_peered":"2024-09-19T04:05:42.336125+0000","last_clean":"2024-09-19T04:05:42.336125+0000","last_became_active":"2024-09-19T04:04:46.354684+0000","last_became_peered":"2024-09-19T04:04:46.354684+0000","last_unstale":"2024-09-19T04:05:42.336125+0000","last_undegraded":"2024-09-19T04:05:42.336125+0000","last_fullsized":"2024-09-19T04:05:42.336125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:33.141690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333184+0000","last_change":"2024-09-19T04:04:37.211772+0000","last_active":"2024-09-19T04:05:42.333184+0000","last_peered":"2024-09-19T04:05:42.333184+0000","last_clean":"2024-09-19T04:05:42.333184+0000","last_became_active":"2024-09-19T04:04:37.211452+0000","last_became_peered":"2024-09-19T04:04:37.211452+0000","last_unstale":"2024-09-19T04:05:42.333184+0000","last_undegraded":"2024-09-19T04:05:42.333184+0000","last_fullsized":"2024-09-19T04:05:42.333184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:57.781481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331808+0000","last_change":"2024-09-19T04:04:39.320791+0000","last_active":"2024-09-19T04:05:42.331808+0000","last_peered":"2024-09-19T04:05:42.331808+0000","last_clean":"2024-09-19T04:05:42.331808+0000","last_became_active":"2024-09-19T04:04:39.320206+0000","last_became_peered":"2024-09-19T04:04:39.320206+0000","last_unstale":"2024-09-19T04:05:42.331808+0000","last_undegraded":"2024-09-19T04:05:42.331808+0000","last_fullsized":"2024-09-19T04:05:42.331808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:17.385974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337258+0000","last_change":"2024-09-19T04:04:46.363375+0000","last_active":"2024-09-19T04:05:42.337258+0000","last_peered":"2024-09-19T04:05:42.337258+0000","last_clean":"2024-09-19T04:05:42.337258+0000","last_became_active":"2024-09-19T04:04:46.362800+0000","last_became_peered":"2024-09-19T04:04:46.362800+0000","last_unstale":"2024-09-19T04:05:42.337258+0000","last_undegraded":"2024-09-19T04:05:42.337258+0000","last_fullsized":"2024-09-19T04:05:42.337258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:55.651507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333276+0000","last_change":"2024-09-19T04:04:37.206218+0000","last_active":"2024-09-19T04:05:42.333276+0000","last_peered":"2024-09-19T04:05:42.333276+0000","last_clean":"2024-09-19T04:05:42.333276+0000","last_became_active":"2024-09-19T04:04:37.205675+0000","last_became_peered":"2024-09-19T04:04:37.205675+0000","last_unstale":"2024-09-19T04:05:42.333276+0000","last_undegraded":"2024-09-19T04:05:42.333276+0000","last_fullsized":"2024-09-19T04:05:42.333276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:27.664960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336567+0000","last_change":"2024-09-19T04:04:39.329667+0000","last_active":"2024-09-19T04:05:42.336567+0000","last_peered":"2024-09-19T04:05:42.336567+0000","last_clean":"2024-09-19T04:05:42.336567+0000","last_became_active":"2024-09-19T04:04:39.328180+0000","last_became_peered":"2024-09-19T04:04:39.328180+0000","last_unstale":"2024-09-19T04:05:42.336567+0000","last_undegraded":"2024-09-19T04:05:42.336567+0000","last_fullsized":"2024-09-19T04:05:42.336567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:59.782472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336861+0000","last_change":"2024-09-19T04:04:46.359432+0000","last_active":"2024-09-19T04:05:42.336861+0000","last_peered":"2024-09-19T04:05:42.336861+0000","last_clean":"2024-09-19T04:05:42.336861+0000","last_became_active":"2024-09-19T04:04:46.358966+0000","last_became_peered":"2024-09-19T04:04:46.358966+0000","last_unstale":"2024-09-19T04:05:42.336861+0000","last_undegraded":"2024-09-19T04:05:42.336861+0000","last_fullsized":"2024-09-19T04:05:42.336861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:41.537926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337468+0000","last_change":"2024-09-19T04:04:39.325084+0000","last_active":"2024-09-19T04:05:42.337468+0000","last_peered":"2024-09-19T04:05:42.337468+0000","last_clean":"2024-09-19T04:05:42.337468+0000","last_became_active":"2024-09-19T04:04:39.324126+0000","last_became_peered":"2024-09-19T04:04:39.324126+0000","last_unstale":"2024-09-19T04:05:42.337468+0000","last_undegraded":"2024-09-19T04:05:42.337468+0000","last_fullsized":"2024-09-19T04:05:42.337468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:13.523889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.359505+0000","last_change":"2024-09-19T04:05:42.359098+0000","last_active":"2024-09-19T04:05:42.359505+0000","last_peered":"2024-09-19T04:05:42.359505+0000","last_clean":"2024-09-19T04:05:42.359505+0000","last_became_active":"2024-09-19T04:05:42.341384+0000","last_became_peered":"2024-09-19T04:05:42.341384+0000","last_unstale":"2024-09-19T04:05:42.359505+0000","last_undegraded":"2024-09-19T04:05:42.359505+0000","last_fullsized":"2024-09-19T04:05:42.359505+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:15:57.418355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336594+0000","last_change":"2024-09-19T04:04:46.374921+0000","last_active":"2024-09-19T04:05:42.336594+0000","last_peered":"2024-09-19T04:05:42.336594+0000","last_clean":"2024-09-19T04:05:42.336594+0000","last_became_active":"2024-09-19T04:04:46.374466+0000","last_became_peered":"2024-09-19T04:04:46.374466+0000","last_unstale":"2024-09-19T04:05:42.336594+0000","last_undegraded":"2024-09-19T04:05:42.336594+0000","last_fullsized":"2024-09-19T04:05:42.336594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:46.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822761+0000","last_change":"2024-09-19T04:04:39.319948+0000","last_active":"2024-09-19T04:05:42.822761+0000","last_peered":"2024-09-19T04:05:42.822761+0000","last_clean":"2024-09-19T04:05:42.822761+0000","last_became_active":"2024-09-19T04:04:39.319675+0000","last_became_peered":"2024-09-19T04:04:39.319675+0000","last_unstale":"2024-09-19T04:05:42.822761+0000","last_undegraded":"2024-09-19T04:05:42.822761+0000","last_fullsized":"2024-09-19T04:05:42.822761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:08.818133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331510+0000","last_change":"2024-09-19T04:04:37.207705+0000","last_active":"2024-09-19T04:05:42.331510+0000","last_peered":"2024-09-19T04:05:42.331510+0000","last_clean":"2024-09-19T04:05:42.331510+0000","last_became_active":"2024-09-19T04:04:37.207251+0000","last_became_peered":"2024-09-19T04:04:37.207251+0000","last_unstale":"2024-09-19T04:05:42.331510+0000","last_undegraded":"2024-09-19T04:05:42.331510+0000","last_fullsized":"2024-09-19T04:05:42.331510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:20.835004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336728+0000","last_change":"2024-09-19T04:04:46.354463+0000","last_active":"2024-09-19T04:05:42.336728+0000","last_peered":"2024-09-19T04:05:42.336728+0000","last_clean":"2024-09-19T04:05:42.336728+0000","last_became_active":"2024-09-19T04:04:46.353754+0000","last_became_peered":"2024-09-19T04:04:46.353754+0000","last_unstale":"2024-09-19T04:05:42.336728+0000","last_undegraded":"2024-09-19T04:05:42.336728+0000","last_fullsized":"2024-09-19T04:05:42.336728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:09.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333676+0000","last_change":"2024-09-19T04:04:39.322727+0000","last_active":"2024-09-19T04:05:42.333676+0000","last_peered":"2024-09-19T04:05:42.333676+0000","last_clean":"2024-09-19T04:05:42.333676+0000","last_became_active":"2024-09-19T04:04:39.320989+0000","last_became_peered":"2024-09-19T04:04:39.320989+0000","last_unstale":"2024-09-19T04:05:42.333676+0000","last_undegraded":"2024-09-19T04:05:42.333676+0000","last_fullsized":"2024-09-19T04:05:42.333676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:06.536540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822964+0000","last_change":"2024-09-19T04:04:37.213602+0000","last_active":"2024-09-19T04:05:42.822964+0000","last_peered":"2024-09-19T04:05:42.822964+0000","last_clean":"2024-09-19T04:05:42.822964+0000","last_became_active":"2024-09-19T04:04:37.213240+0000","last_became_peered":"2024-09-19T04:04:37.213240+0000","last_unstale":"2024-09-19T04:05:42.822964+0000","last_undegraded":"2024-09-19T04:05:42.822964+0000","last_fullsized":"2024-09-19T04:05:42.822964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:09.822393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333269+0000","last_change":"2024-09-19T04:04:46.363921+0000","last_active":"2024-09-19T04:05:42.333269+0000","last_peered":"2024-09-19T04:05:42.333269+0000","last_clean":"2024-09-19T04:05:42.333269+0000","last_became_active":"2024-09-19T04:04:46.363476+0000","last_became_peered":"2024-09-19T04:04:46.363476+0000","last_unstale":"2024-09-19T04:05:42.333269+0000","last_undegraded":"2024-09-19T04:05:42.333269+0000","last_fullsized":"2024-09-19T04:05:42.333269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:32.015782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332715+0000","last_change":"2024-09-19T04:04:39.316814+0000","last_active":"2024-09-19T04:05:42.332715+0000","last_peered":"2024-09-19T04:05:42.332715+0000","last_clean":"2024-09-19T04:05:42.332715+0000","last_became_active":"2024-09-19T04:04:39.316504+0000","last_became_peered":"2024-09-19T04:04:39.316504+0000","last_unstale":"2024-09-19T04:05:42.332715+0000","last_undegraded":"2024-09-19T04:05:42.332715+0000","last_fullsized":"2024-09-19T04:05:42.332715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331734+0000","last_change":"2024-09-19T04:04:37.647260+0000","last_active":"2024-09-19T04:05:42.331734+0000","last_peered":"2024-09-19T04:05:42.331734+0000","last_clean":"2024-09-19T04:05:42.331734+0000","last_became_active":"2024-09-19T04:04:37.646971+0000","last_became_peered":"2024-09-19T04:04:37.646971+0000","last_unstale":"2024-09-19T04:05:42.331734+0000","last_undegraded":"2024-09-19T04:05:42.331734+0000","last_fullsized":"2024-09-19T04:05:42.331734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:07.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.342340+0000","last_change":"2024-09-19T04:05:42.342340+0000","last_active":"2024-09-19T04:05:42.342340+0000","last_peered":"2024-09-19T04:05:42.342340+0000","last_clean":"2024-09-19T04:05:42.342340+0000","last_became_active":"2024-09-19T04:05:42.341861+0000","last_became_peered":"2024-09-19T04:05:42.341861+0000","last_unstale":"2024-09-19T04:05:42.342340+0000","last_undegraded":"2024-09-19T04:05:42.342340+0000","last_fullsized":"2024-09-19T04:05:42.342340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:27.520514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150301+0000","last_change":"2024-09-19T04:04:39.318185+0000","last_active":"2024-09-19T04:05:43.150301+0000","last_peered":"2024-09-19T04:05:43.150301+0000","last_clean":"2024-09-19T04:05:43.150301+0000","last_became_active":"2024-09-19T04:04:39.317885+0000","last_became_peered":"2024-09-19T04:04:39.317885+0000","last_unstale":"2024-09-19T04:05:43.150301+0000","last_undegraded":"2024-09-19T04:05:43.150301+0000","last_fullsized":"2024-09-19T04:05:43.150301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:27.719248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337062+0000","last_change":"2024-09-19T04:04:37.703740+0000","last_active":"2024-09-19T04:05:42.337062+0000","last_peered":"2024-09-19T04:05:42.337062+0000","last_clean":"2024-09-19T04:05:42.337062+0000","last_became_active":"2024-09-19T04:04:37.703414+0000","last_became_peered":"2024-09-19T04:04:37.703414+0000","last_unstale":"2024-09-19T04:05:42.337062+0000","last_undegraded":"2024-09-19T04:05:42.337062+0000","last_fullsized":"2024-09-19T04:05:42.337062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:19.653559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331302+0000","last_change":"2024-09-19T04:04:46.354964+0000","last_active":"2024-09-19T04:05:42.331302+0000","last_peered":"2024-09-19T04:05:42.331302+0000","last_clean":"2024-09-19T04:05:42.331302+0000","last_became_active":"2024-09-19T04:04:46.354494+0000","last_became_peered":"2024-09-19T04:04:46.354494+0000","last_unstale":"2024-09-19T04:05:42.331302+0000","last_undegraded":"2024-09-19T04:05:42.331302+0000","last_fullsized":"2024-09-19T04:05:42.331302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:11.246134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.823151+0000","last_change":"2024-09-19T04:04:39.315811+0000","last_active":"2024-09-19T04:05:42.823151+0000","last_peered":"2024-09-19T04:05:42.823151+0000","last_clean":"2024-09-19T04:05:42.823151+0000","last_became_active":"2024-09-19T04:04:39.315466+0000","last_became_peered":"2024-09-19T04:04:39.315466+0000","last_unstale":"2024-09-19T04:05:42.823151+0000","last_undegraded":"2024-09-19T04:05:42.823151+0000","last_fullsized":"2024-09-19T04:05:42.823151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:17.414765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333299+0000","last_change":"2024-09-19T04:04:37.204206+0000","last_active":"2024-09-19T04:05:42.333299+0000","last_peered":"2024-09-19T04:05:42.333299+0000","last_clean":"2024-09-19T04:05:42.333299+0000","last_became_active":"2024-09-19T04:04:37.203845+0000","last_became_peered":"2024-09-19T04:04:37.203845+0000","last_unstale":"2024-09-19T04:05:42.333299+0000","last_undegraded":"2024-09-19T04:05:42.333299+0000","last_fullsized":"2024-09-19T04:05:42.333299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:52.868643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822240+0000","last_change":"2024-09-19T04:04:46.375541+0000","last_active":"2024-09-19T04:05:42.822240+0000","last_peered":"2024-09-19T04:05:42.822240+0000","last_clean":"2024-09-19T04:05:42.822240+0000","last_became_active":"2024-09-19T04:04:46.375142+0000","last_became_peered":"2024-09-19T04:04:46.375142+0000","last_unstale":"2024-09-19T04:05:42.822240+0000","last_undegraded":"2024-09-19T04:05:42.822240+0000","last_fullsized":"2024-09-19T04:05:42.822240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.889459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464943+0000","last_change":"2024-09-19T04:04:39.324094+0000","last_active":"2024-09-19T04:05:42.464943+0000","last_peered":"2024-09-19T04:05:42.464943+0000","last_clean":"2024-09-19T04:05:42.464943+0000","last_became_active":"2024-09-19T04:04:39.322897+0000","last_became_peered":"2024-09-19T04:04:39.322897+0000","last_unstale":"2024-09-19T04:05:42.464943+0000","last_undegraded":"2024-09-19T04:05:42.464943+0000","last_fullsized":"2024-09-19T04:05:42.464943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:07.773110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333314+0000","last_change":"2024-09-19T04:04:37.647686+0000","last_active":"2024-09-19T04:05:42.333314+0000","last_peered":"2024-09-19T04:05:42.333314+0000","last_clean":"2024-09-19T04:05:42.333314+0000","last_became_active":"2024-09-19T04:04:37.646979+0000","last_became_peered":"2024-09-19T04:04:37.646979+0000","last_unstale":"2024-09-19T04:05:42.333314+0000","last_undegraded":"2024-09-19T04:05:42.333314+0000","last_fullsized":"2024-09-19T04:05:42.333314+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:48.063075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335982+0000","last_change":"2024-09-19T04:04:46.356078+0000","last_active":"2024-09-19T04:05:42.335982+0000","last_peered":"2024-09-19T04:05:42.335982+0000","last_clean":"2024-09-19T04:05:42.335982+0000","last_became_active":"2024-09-19T04:04:46.355630+0000","last_became_peered":"2024-09-19T04:04:46.355630+0000","last_unstale":"2024-09-19T04:05:42.335982+0000","last_undegraded":"2024-09-19T04:05:42.335982+0000","last_fullsized":"2024-09-19T04:05:42.335982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:45.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334352+0000","last_change":"2024-09-19T04:04:39.311067+0000","last_active":"2024-09-19T04:05:42.334352+0000","last_peered":"2024-09-19T04:05:42.334352+0000","last_clean":"2024-09-19T04:05:42.334352+0000","last_became_active":"2024-09-19T04:04:39.310725+0000","last_became_peered":"2024-09-19T04:04:39.310725+0000","last_unstale":"2024-09-19T04:05:42.334352+0000","last_undegraded":"2024-09-19T04:05:42.334352+0000","last_fullsized":"2024-09-19T04:05:42.334352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:47:11.488121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333056+0000","last_change":"2024-09-19T04:04:37.648335+0000","last_active":"2024-09-19T04:05:42.333056+0000","last_peered":"2024-09-19T04:05:42.333056+0000","last_clean":"2024-09-19T04:05:42.333056+0000","last_became_active":"2024-09-19T04:04:37.648000+0000","last_became_peered":"2024-09-19T04:04:37.648000+0000","last_unstale":"2024-09-19T04:05:42.333056+0000","last_undegraded":"2024-09-19T04:05:42.333056+0000","last_fullsized":"2024-09-19T04:05:42.333056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:16.102728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332783+0000","last_change":"2024-09-19T04:04:46.375653+0000","last_active":"2024-09-19T04:05:42.332783+0000","last_peered":"2024-09-19T04:05:42.332783+0000","last_clean":"2024-09-19T04:05:42.332783+0000","last_became_active":"2024-09-19T04:04:46.375132+0000","last_became_peered":"2024-09-19T04:04:46.375132+0000","last_unstale":"2024-09-19T04:05:42.332783+0000","last_undegraded":"2024-09-19T04:05:42.332783+0000","last_fullsized":"2024-09-19T04:05:42.332783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:03.332697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334076+0000","last_change":"2024-09-19T04:04:39.318155+0000","last_active":"2024-09-19T04:05:42.334076+0000","last_peered":"2024-09-19T04:05:42.334076+0000","last_clean":"2024-09-19T04:05:42.334076+0000","last_became_active":"2024-09-19T04:04:39.316710+0000","last_became_peered":"2024-09-19T04:04:39.316710+0000","last_unstale":"2024-09-19T04:05:42.334076+0000","last_undegraded":"2024-09-19T04:05:42.334076+0000","last_fullsized":"2024-09-19T04:05:42.334076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:59.084523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":95,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:13.675552+0000","last_change":"2024-09-19T04:04:37.210695+0000","last_active":"2024-09-19T04:06:13.675552+0000","last_peered":"2024-09-19T04:06:13.675552+0000","last_clean":"2024-09-19T04:06:13.675552+0000","last_became_active":"2024-09-19T04:04:37.210334+0000","last_became_peered":"2024-09-19T04:04:37.210334+0000","last_unstale":"2024-09-19T04:06:13.675552+0000","last_undegraded":"2024-09-19T04:06:13.675552+0000","last_fullsized":"2024-09-19T04:06:13.675552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:12.545248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464242+0000","last_change":"2024-09-19T04:04:46.358130+0000","last_active":"2024-09-19T04:05:42.464242+0000","last_peered":"2024-09-19T04:05:42.464242+0000","last_clean":"2024-09-19T04:05:42.464242+0000","last_became_active":"2024-09-19T04:04:46.356810+0000","last_became_peered":"2024-09-19T04:04:46.356810+0000","last_unstale":"2024-09-19T04:05:42.464242+0000","last_undegraded":"2024-09-19T04:05:42.464242+0000","last_fullsized":"2024-09-19T04:05:42.464242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:49.884478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335465+0000","last_change":"2024-09-19T04:04:39.325866+0000","last_active":"2024-09-19T04:05:42.335465+0000","last_peered":"2024-09-19T04:05:42.335465+0000","last_clean":"2024-09-19T04:05:42.335465+0000","last_became_active":"2024-09-19T04:04:39.325473+0000","last_became_peered":"2024-09-19T04:04:39.325473+0000","last_unstale":"2024-09-19T04:05:42.335465+0000","last_undegraded":"2024-09-19T04:05:42.335465+0000","last_fullsized":"2024-09-19T04:05:42.335465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:30.036704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:14.676349+0000","last_change":"2024-09-19T04:04:37.647310+0000","last_active":"2024-09-19T04:06:14.676349+0000","last_peered":"2024-09-19T04:06:14.676349+0000","last_clean":"2024-09-19T04:06:14.676349+0000","last_became_active":"2024-09-19T04:04:37.646903+0000","last_became_peered":"2024-09-19T04:04:37.646903+0000","last_unstale":"2024-09-19T04:06:14.676349+0000","last_undegraded":"2024-09-19T04:06:14.676349+0000","last_fullsized":"2024-09-19T04:06:14.676349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:20.097472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464406+0000","last_change":"2024-09-19T04:04:46.375261+0000","last_active":"2024-09-19T04:05:42.464406+0000","last_peered":"2024-09-19T04:05:42.464406+0000","last_clean":"2024-09-19T04:05:42.464406+0000","last_became_active":"2024-09-19T04:04:46.374489+0000","last_became_peered":"2024-09-19T04:04:46.374489+0000","last_unstale":"2024-09-19T04:05:42.464406+0000","last_undegraded":"2024-09-19T04:05:42.464406+0000","last_fullsized":"2024-09-19T04:05:42.464406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:05.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464375+0000","last_change":"2024-09-19T04:04:39.322265+0000","last_active":"2024-09-19T04:05:42.464375+0000","last_peered":"2024-09-19T04:05:42.464375+0000","last_clean":"2024-09-19T04:05:42.464375+0000","last_became_active":"2024-09-19T04:04:39.321938+0000","last_became_peered":"2024-09-19T04:04:39.321938+0000","last_unstale":"2024-09-19T04:05:42.464375+0000","last_undegraded":"2024-09-19T04:05:42.464375+0000","last_fullsized":"2024-09-19T04:05:42.464375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:19.220661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.339577+0000","last_change":"2024-09-19T04:04:37.211330+0000","last_active":"2024-09-19T04:05:42.339577+0000","last_peered":"2024-09-19T04:05:42.339577+0000","last_clean":"2024-09-19T04:05:42.339577+0000","last_became_active":"2024-09-19T04:04:37.210788+0000","last_became_peered":"2024-09-19T04:04:37.210788+0000","last_unstale":"2024-09-19T04:05:42.339577+0000","last_undegraded":"2024-09-19T04:05:42.339577+0000","last_fullsized":"2024-09-19T04:05:42.339577+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:43.169897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.465969+0000","last_change":"2024-09-19T04:05:42.465969+0000","last_active":"2024-09-19T04:05:42.465969+0000","last_peered":"2024-09-19T04:05:42.465969+0000","last_clean":"2024-09-19T04:05:42.465969+0000","last_became_active":"2024-09-19T04:05:42.465621+0000","last_became_peered":"2024-09-19T04:05:42.465621+0000","last_unstale":"2024-09-19T04:05:42.465969+0000","last_undegraded":"2024-09-19T04:05:42.465969+0000","last_fullsized":"2024-09-19T04:05:42.465969+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:29.802141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,10],"acting":[5,1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.823049+0000","last_change":"2024-09-19T04:04:39.329480+0000","last_active":"2024-09-19T04:05:42.823049+0000","last_peered":"2024-09-19T04:05:42.823049+0000","last_clean":"2024-09-19T04:05:42.823049+0000","last_became_active":"2024-09-19T04:04:39.329008+0000","last_became_peered":"2024-09-19T04:04:39.329008+0000","last_unstale":"2024-09-19T04:05:42.823049+0000","last_undegraded":"2024-09-19T04:05:42.823049+0000","last_fullsized":"2024-09-19T04:05:42.823049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:06.782579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337719+0000","last_change":"2024-09-19T04:04:37.216224+0000","last_active":"2024-09-19T04:05:42.337719+0000","last_peered":"2024-09-19T04:05:42.337719+0000","last_clean":"2024-09-19T04:05:42.337719+0000","last_became_active":"2024-09-19T04:04:37.215846+0000","last_became_peered":"2024-09-19T04:04:37.215846+0000","last_unstale":"2024-09-19T04:05:42.337719+0000","last_undegraded":"2024-09-19T04:05:42.337719+0000","last_fullsized":"2024-09-19T04:05:42.337719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:57.829694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.339830+0000","last_change":"2024-09-19T04:03:25.869634+0000","last_active":"2024-09-19T04:05:42.339830+0000","last_peered":"2024-09-19T04:05:42.339830+0000","last_clean":"2024-09-19T04:05:42.339830+0000","last_became_active":"2024-09-19T04:03:25.869250+0000","last_became_peered":"2024-09-19T04:03:25.869250+0000","last_unstale":"2024-09-19T04:05:42.339830+0000","last_undegraded":"2024-09-19T04:05:42.339830+0000","last_fullsized":"2024-09-19T04:05:42.339830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_clean_scrub_stamp":"2024-09-19T03:59:59.253417+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:51.956932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338108+0000","last_change":"2024-09-19T04:04:46.357123+0000","last_active":"2024-09-19T04:05:42.338108+0000","last_peered":"2024-09-19T04:05:42.338108+0000","last_clean":"2024-09-19T04:05:42.338108+0000","last_became_active":"2024-09-19T04:04:46.356671+0000","last_became_peered":"2024-09-19T04:04:46.356671+0000","last_unstale":"2024-09-19T04:05:42.338108+0000","last_undegraded":"2024-09-19T04:05:42.338108+0000","last_fullsized":"2024-09-19T04:05:42.338108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:09.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338394+0000","last_change":"2024-09-19T04:04:39.322487+0000","last_active":"2024-09-19T04:05:42.338394+0000","last_peered":"2024-09-19T04:05:42.338394+0000","last_clean":"2024-09-19T04:05:42.338394+0000","last_became_active":"2024-09-19T04:04:39.321994+0000","last_became_peered":"2024-09-19T04:04:39.321994+0000","last_unstale":"2024-09-19T04:05:42.338394+0000","last_undegraded":"2024-09-19T04:05:42.338394+0000","last_fullsized":"2024-09-19T04:05:42.338394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:56.977405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:11.676946+0000","last_change":"2024-09-19T04:04:37.703452+0000","last_active":"2024-09-19T04:06:11.676946+0000","last_peered":"2024-09-19T04:06:11.676946+0000","last_clean":"2024-09-19T04:06:11.676946+0000","last_became_active":"2024-09-19T04:04:37.703048+0000","last_became_peered":"2024-09-19T04:04:37.703048+0000","last_unstale":"2024-09-19T04:06:11.676946+0000","last_undegraded":"2024-09-19T04:06:11.676946+0000","last_fullsized":"2024-09-19T04:06:11.676946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:16:29.051617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331703+0000","last_change":"2024-09-19T04:04:46.361229+0000","last_active":"2024-09-19T04:05:42.331703+0000","last_peered":"2024-09-19T04:05:42.331703+0000","last_clean":"2024-09-19T04:05:42.331703+0000","last_became_active":"2024-09-19T04:04:46.360729+0000","last_became_peered":"2024-09-19T04:04:46.360729+0000","last_unstale":"2024-09-19T04:05:42.331703+0000","last_undegraded":"2024-09-19T04:05:42.331703+0000","last_fullsized":"2024-09-19T04:05:42.331703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:48.022778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337919+0000","last_change":"2024-09-19T04:04:39.328598+0000","last_active":"2024-09-19T04:05:42.337919+0000","last_peered":"2024-09-19T04:05:42.337919+0000","last_clean":"2024-09-19T04:05:42.337919+0000","last_became_active":"2024-09-19T04:04:39.328247+0000","last_became_peered":"2024-09-19T04:04:39.328247+0000","last_unstale":"2024-09-19T04:05:42.337919+0000","last_undegraded":"2024-09-19T04:05:42.337919+0000","last_fullsized":"2024-09-19T04:05:42.337919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:34.916487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336864+0000","last_change":"2024-09-19T04:04:37.212065+0000","last_active":"2024-09-19T04:05:42.336864+0000","last_peered":"2024-09-19T04:05:42.336864+0000","last_clean":"2024-09-19T04:05:42.336864+0000","last_became_active":"2024-09-19T04:04:37.211520+0000","last_became_peered":"2024-09-19T04:04:37.211520+0000","last_unstale":"2024-09-19T04:05:42.336864+0000","last_undegraded":"2024-09-19T04:05:42.336864+0000","last_fullsized":"2024-09-19T04:05:42.336864+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:53.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.149487+0000","last_change":"2024-09-19T04:04:46.358558+0000","last_active":"2024-09-19T04:05:43.149487+0000","last_peered":"2024-09-19T04:05:43.149487+0000","last_clean":"2024-09-19T04:05:43.149487+0000","last_became_active":"2024-09-19T04:04:46.358158+0000","last_became_peered":"2024-09-19T04:04:46.358158+0000","last_unstale":"2024-09-19T04:05:43.149487+0000","last_undegraded":"2024-09-19T04:05:43.149487+0000","last_fullsized":"2024-09-19T04:05:43.149487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:16.627051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338837+0000","last_change":"2024-09-19T04:04:39.325187+0000","last_active":"2024-09-19T04:05:42.338837+0000","last_peered":"2024-09-19T04:05:42.338837+0000","last_clean":"2024-09-19T04:05:42.338837+0000","last_became_active":"2024-09-19T04:04:39.324644+0000","last_became_peered":"2024-09-19T04:04:39.324644+0000","last_unstale":"2024-09-19T04:05:42.338837+0000","last_undegraded":"2024-09-19T04:05:42.338837+0000","last_fullsized":"2024-09-19T04:05:42.338837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:11.879273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464944+0000","last_change":"2024-09-19T04:04:37.205389+0000","last_active":"2024-09-19T04:05:42.464944+0000","last_peered":"2024-09-19T04:05:42.464944+0000","last_clean":"2024-09-19T04:05:42.464944+0000","last_became_active":"2024-09-19T04:04:37.205117+0000","last_became_peered":"2024-09-19T04:04:37.205117+0000","last_unstale":"2024-09-19T04:05:42.464944+0000","last_undegraded":"2024-09-19T04:05:42.464944+0000","last_fullsized":"2024-09-19T04:05:42.464944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:49.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.342246+0000","last_change":"2024-09-19T04:05:42.342246+0000","last_active":"2024-09-19T04:05:42.342246+0000","last_peered":"2024-09-19T04:05:42.342246+0000","last_clean":"2024-09-19T04:05:42.342246+0000","last_became_active":"2024-09-19T04:05:42.341628+0000","last_became_peered":"2024-09-19T04:05:42.341628+0000","last_unstale":"2024-09-19T04:05:42.342246+0000","last_undegraded":"2024-09-19T04:05:42.342246+0000","last_fullsized":"2024-09-19T04:05:42.342246+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:16.637647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333643+0000","last_change":"2024-09-19T04:04:39.328841+0000","last_active":"2024-09-19T04:05:42.333643+0000","last_peered":"2024-09-19T04:05:42.333643+0000","last_clean":"2024-09-19T04:05:42.333643+0000","last_became_active":"2024-09-19T04:04:39.328470+0000","last_became_peered":"2024-09-19T04:04:39.328470+0000","last_unstale":"2024-09-19T04:05:42.333643+0000","last_undegraded":"2024-09-19T04:05:42.333643+0000","last_fullsized":"2024-09-19T04:05:42.333643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:17.701469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.353928+0000","last_change":"2024-09-19T04:05:42.353456+0000","last_active":"2024-09-19T04:05:42.353928+0000","last_peered":"2024-09-19T04:05:42.353928+0000","last_clean":"2024-09-19T04:05:42.353928+0000","last_became_active":"2024-09-19T04:05:42.340710+0000","last_became_peered":"2024-09-19T04:05:42.340710+0000","last_unstale":"2024-09-19T04:05:42.353928+0000","last_undegraded":"2024-09-19T04:05:42.353928+0000","last_fullsized":"2024-09-19T04:05:42.353928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:45.262693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335416+0000","last_change":"2024-09-19T04:04:46.361199+0000","last_active":"2024-09-19T04:05:42.335416+0000","last_peered":"2024-09-19T04:05:42.335416+0000","last_clean":"2024-09-19T04:05:42.335416+0000","last_became_active":"2024-09-19T04:04:46.360639+0000","last_became_peered":"2024-09-19T04:04:46.360639+0000","last_unstale":"2024-09-19T04:05:42.335416+0000","last_undegraded":"2024-09-19T04:05:42.335416+0000","last_fullsized":"2024-09-19T04:05:42.335416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:50.185760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335552+0000","last_change":"2024-09-19T04:04:39.328529+0000","last_active":"2024-09-19T04:05:42.335552+0000","last_peered":"2024-09-19T04:05:42.335552+0000","last_clean":"2024-09-19T04:05:42.335552+0000","last_became_active":"2024-09-19T04:04:39.328174+0000","last_became_peered":"2024-09-19T04:04:39.328174+0000","last_unstale":"2024-09-19T04:05:42.335552+0000","last_undegraded":"2024-09-19T04:05:42.335552+0000","last_fullsized":"2024-09-19T04:05:42.335552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:02.503672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335126+0000","last_change":"2024-09-19T04:04:37.647919+0000","last_active":"2024-09-19T04:05:42.335126+0000","last_peered":"2024-09-19T04:05:42.335126+0000","last_clean":"2024-09-19T04:05:42.335126+0000","last_became_active":"2024-09-19T04:04:37.647305+0000","last_became_peered":"2024-09-19T04:04:37.647305+0000","last_unstale":"2024-09-19T04:05:42.335126+0000","last_undegraded":"2024-09-19T04:05:42.335126+0000","last_fullsized":"2024-09-19T04:05:42.335126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:07.035465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464291+0000","last_change":"2024-09-19T04:04:46.360362+0000","last_active":"2024-09-19T04:05:42.464291+0000","last_peered":"2024-09-19T04:05:42.464291+0000","last_clean":"2024-09-19T04:05:42.464291+0000","last_became_active":"2024-09-19T04:04:46.359470+0000","last_became_peered":"2024-09-19T04:04:46.359470+0000","last_unstale":"2024-09-19T04:05:42.464291+0000","last_undegraded":"2024-09-19T04:05:42.464291+0000","last_fullsized":"2024-09-19T04:05:42.464291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:43.804073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.339402+0000","last_change":"2024-09-19T04:04:39.326090+0000","last_active":"2024-09-19T04:05:42.339402+0000","last_peered":"2024-09-19T04:05:42.339402+0000","last_clean":"2024-09-19T04:05:42.339402+0000","last_became_active":"2024-09-19T04:04:39.325751+0000","last_became_peered":"2024-09-19T04:04:39.325751+0000","last_unstale":"2024-09-19T04:05:42.339402+0000","last_undegraded":"2024-09-19T04:05:42.339402+0000","last_fullsized":"2024-09-19T04:05:42.339402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:25.357467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:13.675638+0000","last_change":"2024-09-19T04:04:37.215107+0000","last_active":"2024-09-19T04:06:13.675638+0000","last_peered":"2024-09-19T04:06:13.675638+0000","last_clean":"2024-09-19T04:06:13.675638+0000","last_became_active":"2024-09-19T04:04:37.214693+0000","last_became_peered":"2024-09-19T04:04:37.214693+0000","last_unstale":"2024-09-19T04:06:13.675638+0000","last_undegraded":"2024-09-19T04:06:13.675638+0000","last_fullsized":"2024-09-19T04:06:13.675638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:56.535316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333298+0000","last_change":"2024-09-19T04:04:46.363625+0000","last_active":"2024-09-19T04:05:42.333298+0000","last_peered":"2024-09-19T04:05:42.333298+0000","last_clean":"2024-09-19T04:05:42.333298+0000","last_became_active":"2024-09-19T04:04:46.363207+0000","last_became_peered":"2024-09-19T04:04:46.363207+0000","last_unstale":"2024-09-19T04:05:42.333298+0000","last_undegraded":"2024-09-19T04:05:42.333298+0000","last_fullsized":"2024-09-19T04:05:42.333298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:30.251168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.466411+0000","last_change":"2024-09-19T04:05:42.466411+0000","last_active":"2024-09-19T04:05:42.466411+0000","last_peered":"2024-09-19T04:05:42.466411+0000","last_clean":"2024-09-19T04:05:42.466411+0000","last_became_active":"2024-09-19T04:05:42.465999+0000","last_became_peered":"2024-09-19T04:05:42.465999+0000","last_unstale":"2024-09-19T04:05:42.466411+0000","last_undegraded":"2024-09-19T04:05:42.466411+0000","last_fullsized":"2024-09-19T04:05:42.466411+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:51.289726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336089+0000","last_change":"2024-09-19T04:04:37.212103+0000","last_active":"2024-09-19T04:05:42.336089+0000","last_peered":"2024-09-19T04:05:42.336089+0000","last_clean":"2024-09-19T04:05:42.336089+0000","last_became_active":"2024-09-19T04:04:37.211739+0000","last_became_peered":"2024-09-19T04:04:37.211739+0000","last_unstale":"2024-09-19T04:05:42.336089+0000","last_undegraded":"2024-09-19T04:05:42.336089+0000","last_fullsized":"2024-09-19T04:05:42.336089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:06.588194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333377+0000","last_change":"2024-09-19T04:04:46.361975+0000","last_active":"2024-09-19T04:05:42.333377+0000","last_peered":"2024-09-19T04:05:42.333377+0000","last_clean":"2024-09-19T04:05:42.333377+0000","last_became_active":"2024-09-19T04:04:46.361492+0000","last_became_peered":"2024-09-19T04:04:46.361492+0000","last_unstale":"2024-09-19T04:05:42.333377+0000","last_undegraded":"2024-09-19T04:05:42.333377+0000","last_fullsized":"2024-09-19T04:05:42.333377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:52.723836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464862+0000","last_change":"2024-09-19T04:04:39.317756+0000","last_active":"2024-09-19T04:05:42.464862+0000","last_peered":"2024-09-19T04:05:42.464862+0000","last_clean":"2024-09-19T04:05:42.464862+0000","last_became_active":"2024-09-19T04:04:39.317322+0000","last_became_peered":"2024-09-19T04:04:39.317322+0000","last_unstale":"2024-09-19T04:05:42.464862+0000","last_undegraded":"2024-09-19T04:05:42.464862+0000","last_fullsized":"2024-09-19T04:05:42.464862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:26.291574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331895+0000","last_change":"2024-09-19T04:04:37.213892+0000","last_active":"2024-09-19T04:05:42.331895+0000","last_peered":"2024-09-19T04:05:42.331895+0000","last_clean":"2024-09-19T04:05:42.331895+0000","last_became_active":"2024-09-19T04:04:37.213350+0000","last_became_peered":"2024-09-19T04:04:37.213350+0000","last_unstale":"2024-09-19T04:05:42.331895+0000","last_undegraded":"2024-09-19T04:05:42.331895+0000","last_fullsized":"2024-09-19T04:05:42.331895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:40.065717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336452+0000","last_change":"2024-09-19T04:04:46.360566+0000","last_active":"2024-09-19T04:05:42.336452+0000","last_peered":"2024-09-19T04:05:42.336452+0000","last_clean":"2024-09-19T04:05:42.336452+0000","last_became_active":"2024-09-19T04:04:46.360143+0000","last_became_peered":"2024-09-19T04:04:46.360143+0000","last_unstale":"2024-09-19T04:05:42.336452+0000","last_undegraded":"2024-09-19T04:05:42.336452+0000","last_fullsized":"2024-09-19T04:05:42.336452+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:59.997088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332966+0000","last_change":"2024-09-19T04:04:39.318363+0000","last_active":"2024-09-19T04:05:42.332966+0000","last_peered":"2024-09-19T04:05:42.332966+0000","last_clean":"2024-09-19T04:05:42.332966+0000","last_became_active":"2024-09-19T04:04:39.318003+0000","last_became_peered":"2024-09-19T04:04:39.318003+0000","last_unstale":"2024-09-19T04:05:42.332966+0000","last_undegraded":"2024-09-19T04:05:42.332966+0000","last_fullsized":"2024-09-19T04:05:42.332966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:54.128795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":196,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:12.385843+0000","last_change":"2024-09-19T04:05:42.349944+0000","last_active":"2024-09-19T04:06:12.385843+0000","last_peered":"2024-09-19T04:06:12.385843+0000","last_clean":"2024-09-19T04:06:12.385843+0000","last_became_active":"2024-09-19T04:05:42.339605+0000","last_became_peered":"2024-09-19T04:05:42.339605+0000","last_unstale":"2024-09-19T04:06:12.385843+0000","last_undegraded":"2024-09-19T04:06:12.385843+0000","last_fullsized":"2024-09-19T04:06:12.385843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:17.508316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":116,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":60938,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332887+0000","last_change":"2024-09-19T04:04:46.362967+0000","last_active":"2024-09-19T04:05:42.332887+0000","last_peered":"2024-09-19T04:05:42.332887+0000","last_clean":"2024-09-19T04:05:42.332887+0000","last_became_active":"2024-09-19T04:04:46.362529+0000","last_became_peered":"2024-09-19T04:04:46.362529+0000","last_unstale":"2024-09-19T04:05:42.332887+0000","last_undegraded":"2024-09-19T04:05:42.332887+0000","last_fullsized":"2024-09-19T04:05:42.332887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:01.821697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331910+0000","last_change":"2024-09-19T04:04:39.323499+0000","last_active":"2024-09-19T04:05:42.331910+0000","last_peered":"2024-09-19T04:05:42.331910+0000","last_clean":"2024-09-19T04:05:42.331910+0000","last_became_active":"2024-09-19T04:04:39.323076+0000","last_became_peered":"2024-09-19T04:04:39.323076+0000","last_unstale":"2024-09-19T04:05:42.331910+0000","last_undegraded":"2024-09-19T04:05:42.331910+0000","last_fullsized":"2024-09-19T04:05:42.331910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:10.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336750+0000","last_change":"2024-09-19T04:04:37.647927+0000","last_active":"2024-09-19T04:05:42.336750+0000","last_peered":"2024-09-19T04:05:42.336750+0000","last_clean":"2024-09-19T04:05:42.336750+0000","last_became_active":"2024-09-19T04:04:37.647279+0000","last_became_peered":"2024-09-19T04:04:37.647279+0000","last_unstale":"2024-09-19T04:05:42.336750+0000","last_undegraded":"2024-09-19T04:05:42.336750+0000","last_fullsized":"2024-09-19T04:05:42.336750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:05:50.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337020+0000","last_change":"2024-09-19T04:04:46.363562+0000","last_active":"2024-09-19T04:05:42.337020+0000","last_peered":"2024-09-19T04:05:42.337020+0000","last_clean":"2024-09-19T04:05:42.337020+0000","last_became_active":"2024-09-19T04:04:46.362915+0000","last_became_peered":"2024-09-19T04:04:46.362915+0000","last_unstale":"2024-09-19T04:05:42.337020+0000","last_undegraded":"2024-09-19T04:05:42.337020+0000","last_fullsized":"2024-09-19T04:05:42.337020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:08.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150482+0000","last_change":"2024-09-19T04:04:39.329034+0000","last_active":"2024-09-19T04:05:43.150482+0000","last_peered":"2024-09-19T04:05:43.150482+0000","last_clean":"2024-09-19T04:05:43.150482+0000","last_became_active":"2024-09-19T04:04:39.328692+0000","last_became_peered":"2024-09-19T04:04:39.328692+0000","last_unstale":"2024-09-19T04:05:43.150482+0000","last_undegraded":"2024-09-19T04:05:43.150482+0000","last_fullsized":"2024-09-19T04:05:43.150482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:42.903089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338635+0000","last_change":"2024-09-19T04:04:37.212393+0000","last_active":"2024-09-19T04:05:42.338635+0000","last_peered":"2024-09-19T04:05:42.338635+0000","last_clean":"2024-09-19T04:05:42.338635+0000","last_became_active":"2024-09-19T04:04:37.212013+0000","last_became_peered":"2024-09-19T04:04:37.212013+0000","last_unstale":"2024-09-19T04:05:42.338635+0000","last_undegraded":"2024-09-19T04:05:42.338635+0000","last_fullsized":"2024-09-19T04:05:42.338635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:37.981075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.151378+0000","last_change":"2024-09-19T04:05:43.151378+0000","last_active":"2024-09-19T04:05:43.151378+0000","last_peered":"2024-09-19T04:05:43.151378+0000","last_clean":"2024-09-19T04:05:43.151378+0000","last_became_active":"2024-09-19T04:05:43.150907+0000","last_became_peered":"2024-09-19T04:05:43.150907+0000","last_unstale":"2024-09-19T04:05:43.151378+0000","last_undegraded":"2024-09-19T04:05:43.151378+0000","last_fullsized":"2024-09-19T04:05:43.151378+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:09.542371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332236+0000","last_change":"2024-09-19T04:04:39.319705+0000","last_active":"2024-09-19T04:05:42.332236+0000","last_peered":"2024-09-19T04:05:42.332236+0000","last_clean":"2024-09-19T04:05:42.332236+0000","last_became_active":"2024-09-19T04:04:39.318065+0000","last_became_peered":"2024-09-19T04:04:39.318065+0000","last_unstale":"2024-09-19T04:05:42.332236+0000","last_undegraded":"2024-09-19T04:05:42.332236+0000","last_fullsized":"2024-09-19T04:05:42.332236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:15.630927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:12.677758+0000","last_change":"2024-09-19T04:04:37.215192+0000","last_active":"2024-09-19T04:06:12.677758+0000","last_peered":"2024-09-19T04:06:12.677758+0000","last_clean":"2024-09-19T04:06:12.677758+0000","last_became_active":"2024-09-19T04:04:37.214872+0000","last_became_peered":"2024-09-19T04:04:37.214872+0000","last_unstale":"2024-09-19T04:06:12.677758+0000","last_undegraded":"2024-09-19T04:06:12.677758+0000","last_fullsized":"2024-09-19T04:06:12.677758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:18.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822243+0000","last_change":"2024-09-19T04:04:46.358113+0000","last_active":"2024-09-19T04:05:42.822243+0000","last_peered":"2024-09-19T04:05:42.822243+0000","last_clean":"2024-09-19T04:05:42.822243+0000","last_became_active":"2024-09-19T04:04:46.356604+0000","last_became_peered":"2024-09-19T04:04:46.356604+0000","last_unstale":"2024-09-19T04:05:42.822243+0000","last_undegraded":"2024-09-19T04:05:42.822243+0000","last_fullsized":"2024-09-19T04:05:42.822243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:38.147738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.149511+0000","last_change":"2024-09-19T04:04:46.354516+0000","last_active":"2024-09-19T04:05:43.149511+0000","last_peered":"2024-09-19T04:05:43.149511+0000","last_clean":"2024-09-19T04:05:43.149511+0000","last_became_active":"2024-09-19T04:04:46.354034+0000","last_became_peered":"2024-09-19T04:04:46.354034+0000","last_unstale":"2024-09-19T04:05:43.149511+0000","last_undegraded":"2024-09-19T04:05:43.149511+0000","last_fullsized":"2024-09-19T04:05:43.149511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:21.946746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332777+0000","last_change":"2024-09-19T04:04:39.311324+0000","last_active":"2024-09-19T04:05:42.332777+0000","last_peered":"2024-09-19T04:05:42.332777+0000","last_clean":"2024-09-19T04:05:42.332777+0000","last_became_active":"2024-09-19T04:04:39.310977+0000","last_became_peered":"2024-09-19T04:04:39.310977+0000","last_unstale":"2024-09-19T04:05:42.332777+0000","last_undegraded":"2024-09-19T04:05:42.332777+0000","last_fullsized":"2024-09-19T04:05:42.332777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:43.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335659+0000","last_change":"2024-09-19T04:04:37.212773+0000","last_active":"2024-09-19T04:05:42.335659+0000","last_peered":"2024-09-19T04:05:42.335659+0000","last_clean":"2024-09-19T04:05:42.335659+0000","last_became_active":"2024-09-19T04:04:37.212405+0000","last_became_peered":"2024-09-19T04:04:37.212405+0000","last_unstale":"2024-09-19T04:05:42.335659+0000","last_undegraded":"2024-09-19T04:05:42.335659+0000","last_fullsized":"2024-09-19T04:05:42.335659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:29.020122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334347+0000","last_change":"2024-09-19T04:04:46.363052+0000","last_active":"2024-09-19T04:05:42.334347+0000","last_peered":"2024-09-19T04:05:42.334347+0000","last_clean":"2024-09-19T04:05:42.334347+0000","last_became_active":"2024-09-19T04:04:46.362588+0000","last_became_peered":"2024-09-19T04:04:46.362588+0000","last_unstale":"2024-09-19T04:05:42.334347+0000","last_undegraded":"2024-09-19T04:05:42.334347+0000","last_fullsized":"2024-09-19T04:05:42.334347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:52.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822622+0000","last_change":"2024-09-19T04:04:39.313686+0000","last_active":"2024-09-19T04:05:42.822622+0000","last_peered":"2024-09-19T04:05:42.822622+0000","last_clean":"2024-09-19T04:05:42.822622+0000","last_became_active":"2024-09-19T04:04:39.313386+0000","last_became_peered":"2024-09-19T04:04:39.313386+0000","last_unstale":"2024-09-19T04:05:42.822622+0000","last_undegraded":"2024-09-19T04:05:42.822622+0000","last_fullsized":"2024-09-19T04:05:42.822622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:08.274930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464556+0000","last_change":"2024-09-19T04:04:37.207926+0000","last_active":"2024-09-19T04:05:42.464556+0000","last_peered":"2024-09-19T04:05:42.464556+0000","last_clean":"2024-09-19T04:05:42.464556+0000","last_became_active":"2024-09-19T04:04:37.207538+0000","last_became_peered":"2024-09-19T04:04:37.207538+0000","last_unstale":"2024-09-19T04:05:42.464556+0000","last_undegraded":"2024-09-19T04:05:42.464556+0000","last_fullsized":"2024-09-19T04:05:42.464556+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:29.621638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334223+0000","last_change":"2024-09-19T04:04:37.212829+0000","last_active":"2024-09-19T04:05:42.334223+0000","last_peered":"2024-09-19T04:05:42.334223+0000","last_clean":"2024-09-19T04:05:42.334223+0000","last_became_active":"2024-09-19T04:04:37.212294+0000","last_became_peered":"2024-09-19T04:04:37.212294+0000","last_unstale":"2024-09-19T04:05:42.334223+0000","last_undegraded":"2024-09-19T04:05:42.334223+0000","last_fullsized":"2024-09-19T04:05:42.334223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:34.479708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335826+0000","last_change":"2024-09-19T04:04:39.328622+0000","last_active":"2024-09-19T04:05:42.335826+0000","last_peered":"2024-09-19T04:05:42.335826+0000","last_clean":"2024-09-19T04:05:42.335826+0000","last_became_active":"2024-09-19T04:04:39.328290+0000","last_became_peered":"2024-09-19T04:04:39.328290+0000","last_unstale":"2024-09-19T04:05:42.335826+0000","last_undegraded":"2024-09-19T04:05:42.335826+0000","last_fullsized":"2024-09-19T04:05:42.335826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:04.290767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331562+0000","last_change":"2024-09-19T04:04:46.362853+0000","last_active":"2024-09-19T04:05:42.331562+0000","last_peered":"2024-09-19T04:05:42.331562+0000","last_clean":"2024-09-19T04:05:42.331562+0000","last_became_active":"2024-09-19T04:04:46.362392+0000","last_became_peered":"2024-09-19T04:04:46.362392+0000","last_unstale":"2024-09-19T04:05:42.331562+0000","last_undegraded":"2024-09-19T04:05:42.331562+0000","last_fullsized":"2024-09-19T04:05:42.331562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:40.016502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:06:13.680140+0000","last_change":"2024-09-19T04:04:37.213248+0000","last_active":"2024-09-19T04:06:13.680140+0000","last_peered":"2024-09-19T04:06:13.680140+0000","last_clean":"2024-09-19T04:06:13.680140+0000","last_became_active":"2024-09-19T04:04:37.212639+0000","last_became_peered":"2024-09-19T04:04:37.212639+0000","last_unstale":"2024-09-19T04:06:13.680140+0000","last_undegraded":"2024-09-19T04:06:13.680140+0000","last_fullsized":"2024-09-19T04:06:13.680140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:45:05.917792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333052+0000","last_change":"2024-09-19T04:04:39.330160+0000","last_active":"2024-09-19T04:05:42.333052+0000","last_peered":"2024-09-19T04:05:42.333052+0000","last_clean":"2024-09-19T04:05:42.333052+0000","last_became_active":"2024-09-19T04:04:39.329828+0000","last_became_peered":"2024-09-19T04:04:39.329828+0000","last_unstale":"2024-09-19T04:05:42.333052+0000","last_undegraded":"2024-09-19T04:05:42.333052+0000","last_fullsized":"2024-09-19T04:05:42.333052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:57.712550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332247+0000","last_change":"2024-09-19T04:04:46.360459+0000","last_active":"2024-09-19T04:05:42.332247+0000","last_peered":"2024-09-19T04:05:42.332247+0000","last_clean":"2024-09-19T04:05:42.332247+0000","last_became_active":"2024-09-19T04:04:46.359914+0000","last_became_peered":"2024-09-19T04:04:46.359914+0000","last_unstale":"2024-09-19T04:05:42.332247+0000","last_undegraded":"2024-09-19T04:05:42.332247+0000","last_fullsized":"2024-09-19T04:05:42.332247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:30.380358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333565+0000","last_change":"2024-09-19T04:04:39.324847+0000","last_active":"2024-09-19T04:05:42.333565+0000","last_peered":"2024-09-19T04:05:42.333565+0000","last_clean":"2024-09-19T04:05:42.333565+0000","last_became_active":"2024-09-19T04:04:39.324402+0000","last_became_peered":"2024-09-19T04:04:39.324402+0000","last_unstale":"2024-09-19T04:05:42.333565+0000","last_undegraded":"2024-09-19T04:05:42.333565+0000","last_fullsized":"2024-09-19T04:05:42.333565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:41.996135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.339436+0000","last_change":"2024-09-19T04:05:42.339436+0000","last_active":"2024-09-19T04:05:42.339436+0000","last_peered":"2024-09-19T04:05:42.339436+0000","last_clean":"2024-09-19T04:05:42.339436+0000","last_became_active":"2024-09-19T04:05:42.339048+0000","last_became_peered":"2024-09-19T04:05:42.339048+0000","last_unstale":"2024-09-19T04:05:42.339436+0000","last_undegraded":"2024-09-19T04:05:42.339436+0000","last_fullsized":"2024-09-19T04:05:42.339436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:53.175494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331023+0000","last_change":"2024-09-19T04:04:39.318556+0000","last_active":"2024-09-19T04:05:42.331023+0000","last_peered":"2024-09-19T04:05:42.331023+0000","last_clean":"2024-09-19T04:05:42.331023+0000","last_became_active":"2024-09-19T04:04:39.318200+0000","last_became_peered":"2024-09-19T04:04:39.318200+0000","last_unstale":"2024-09-19T04:05:42.331023+0000","last_undegraded":"2024-09-19T04:05:42.331023+0000","last_fullsized":"2024-09-19T04:05:42.331023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:40.847440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150170+0000","last_change":"2024-09-19T04:04:39.320698+0000","last_active":"2024-09-19T04:05:43.150170+0000","last_peered":"2024-09-19T04:05:43.150170+0000","last_clean":"2024-09-19T04:05:43.150170+0000","last_became_active":"2024-09-19T04:04:39.320419+0000","last_became_peered":"2024-09-19T04:04:39.320419+0000","last_unstale":"2024-09-19T04:05:43.150170+0000","last_undegraded":"2024-09-19T04:05:43.150170+0000","last_fullsized":"2024-09-19T04:05:43.150170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:32.708394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.464888+0000","last_change":"2024-09-19T04:04:39.319886+0000","last_active":"2024-09-19T04:05:42.464888+0000","last_peered":"2024-09-19T04:05:42.464888+0000","last_clean":"2024-09-19T04:05:42.464888+0000","last_became_active":"2024-09-19T04:04:39.319217+0000","last_became_peered":"2024-09-19T04:04:39.319217+0000","last_unstale":"2024-09-19T04:05:42.464888+0000","last_undegraded":"2024-09-19T04:05:42.464888+0000","last_fullsized":"2024-09-19T04:05:42.464888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:38.829608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334219+0000","last_change":"2024-09-19T04:04:39.327639+0000","last_active":"2024-09-19T04:05:42.334219+0000","last_peered":"2024-09-19T04:05:42.334219+0000","last_clean":"2024-09-19T04:05:42.334219+0000","last_became_active":"2024-09-19T04:04:39.327349+0000","last_became_peered":"2024-09-19T04:04:39.327349+0000","last_unstale":"2024-09-19T04:05:42.334219+0000","last_undegraded":"2024-09-19T04:05:42.334219+0000","last_fullsized":"2024-09-19T04:05:42.334219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:31.056609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332017+0000","last_change":"2024-09-19T04:04:39.320253+0000","last_active":"2024-09-19T04:05:42.332017+0000","last_peered":"2024-09-19T04:05:42.332017+0000","last_clean":"2024-09-19T04:05:42.332017+0000","last_became_active":"2024-09-19T04:04:39.319851+0000","last_became_peered":"2024-09-19T04:04:39.319851+0000","last_unstale":"2024-09-19T04:05:42.332017+0000","last_undegraded":"2024-09-19T04:05:42.332017+0000","last_fullsized":"2024-09-19T04:05:42.332017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:53.631216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.823059+0000","last_change":"2024-09-19T04:04:39.322779+0000","last_active":"2024-09-19T04:05:42.823059+0000","last_peered":"2024-09-19T04:05:42.823059+0000","last_clean":"2024-09-19T04:05:42.823059+0000","last_became_active":"2024-09-19T04:04:39.322498+0000","last_became_peered":"2024-09-19T04:04:39.322498+0000","last_unstale":"2024-09-19T04:05:42.823059+0000","last_undegraded":"2024-09-19T04:05:42.823059+0000","last_fullsized":"2024-09-19T04:05:42.823059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:24:11.956002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.335250+0000","last_change":"2024-09-19T04:04:39.322759+0000","last_active":"2024-09-19T04:05:42.335250+0000","last_peered":"2024-09-19T04:05:42.335250+0000","last_clean":"2024-09-19T04:05:42.335250+0000","last_became_active":"2024-09-19T04:04:39.322429+0000","last_became_peered":"2024-09-19T04:04:39.322429+0000","last_unstale":"2024-09-19T04:05:42.335250+0000","last_undegraded":"2024-09-19T04:05:42.335250+0000","last_fullsized":"2024-09-19T04:05:42.335250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:36.069931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822800+0000","last_change":"2024-09-19T04:04:39.320078+0000","last_active":"2024-09-19T04:05:42.822800+0000","last_peered":"2024-09-19T04:05:42.822800+0000","last_clean":"2024-09-19T04:05:42.822800+0000","last_became_active":"2024-09-19T04:04:39.319755+0000","last_became_peered":"2024-09-19T04:04:39.319755+0000","last_unstale":"2024-09-19T04:05:42.822800+0000","last_undegraded":"2024-09-19T04:05:42.822800+0000","last_fullsized":"2024-09-19T04:05:42.822800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:07.651712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331888+0000","last_change":"2024-09-19T04:04:39.319839+0000","last_active":"2024-09-19T04:05:42.331888+0000","last_peered":"2024-09-19T04:05:42.331888+0000","last_clean":"2024-09-19T04:05:42.331888+0000","last_became_active":"2024-09-19T04:04:39.319527+0000","last_became_peered":"2024-09-19T04:04:39.319527+0000","last_unstale":"2024-09-19T04:05:42.331888+0000","last_undegraded":"2024-09-19T04:05:42.331888+0000","last_fullsized":"2024-09-19T04:05:42.331888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:31.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.465177+0000","last_change":"2024-09-19T04:04:39.315975+0000","last_active":"2024-09-19T04:05:42.465177+0000","last_peered":"2024-09-19T04:05:42.465177+0000","last_clean":"2024-09-19T04:05:42.465177+0000","last_became_active":"2024-09-19T04:04:39.315671+0000","last_became_peered":"2024-09-19T04:04:39.315671+0000","last_unstale":"2024-09-19T04:05:42.465177+0000","last_undegraded":"2024-09-19T04:05:42.465177+0000","last_fullsized":"2024-09-19T04:05:42.465177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:16.349853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333449+0000","last_change":"2024-09-19T04:04:39.316278+0000","last_active":"2024-09-19T04:05:42.333449+0000","last_peered":"2024-09-19T04:05:42.333449+0000","last_clean":"2024-09-19T04:05:42.333449+0000","last_became_active":"2024-09-19T04:04:39.315876+0000","last_became_peered":"2024-09-19T04:04:39.315876+0000","last_unstale":"2024-09-19T04:05:42.333449+0000","last_undegraded":"2024-09-19T04:05:42.333449+0000","last_fullsized":"2024-09-19T04:05:42.333449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:23.544405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332393+0000","last_change":"2024-09-19T04:04:39.318952+0000","last_active":"2024-09-19T04:05:42.332393+0000","last_peered":"2024-09-19T04:05:42.332393+0000","last_clean":"2024-09-19T04:05:42.332393+0000","last_became_active":"2024-09-19T04:04:39.318598+0000","last_became_peered":"2024-09-19T04:04:39.318598+0000","last_unstale":"2024-09-19T04:05:42.332393+0000","last_undegraded":"2024-09-19T04:05:42.332393+0000","last_fullsized":"2024-09-19T04:05:42.332393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:12.654611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.333268+0000","last_change":"2024-09-19T04:04:39.311334+0000","last_active":"2024-09-19T04:05:42.333268+0000","last_peered":"2024-09-19T04:05:42.333268+0000","last_clean":"2024-09-19T04:05:42.333268+0000","last_became_active":"2024-09-19T04:04:39.311038+0000","last_became_peered":"2024-09-19T04:04:39.311038+0000","last_unstale":"2024-09-19T04:05:42.333268+0000","last_undegraded":"2024-09-19T04:05:42.333268+0000","last_fullsized":"2024-09-19T04:05:42.333268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:41.655363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336230+0000","last_change":"2024-09-19T04:04:39.319529+0000","last_active":"2024-09-19T04:05:42.336230+0000","last_peered":"2024-09-19T04:05:42.336230+0000","last_clean":"2024-09-19T04:05:42.336230+0000","last_became_active":"2024-09-19T04:04:39.318787+0000","last_became_peered":"2024-09-19T04:04:39.318787+0000","last_unstale":"2024-09-19T04:05:42.336230+0000","last_undegraded":"2024-09-19T04:05:42.336230+0000","last_fullsized":"2024-09-19T04:05:42.336230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.209422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.331537+0000","last_change":"2024-09-19T04:04:39.320320+0000","last_active":"2024-09-19T04:05:42.331537+0000","last_peered":"2024-09-19T04:05:42.331537+0000","last_clean":"2024-09-19T04:05:42.331537+0000","last_became_active":"2024-09-19T04:04:39.319091+0000","last_became_peered":"2024-09-19T04:04:39.319091+0000","last_unstale":"2024-09-19T04:05:42.331537+0000","last_undegraded":"2024-09-19T04:05:42.331537+0000","last_fullsized":"2024-09-19T04:05:42.331537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:39.495079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.149666+0000","last_change":"2024-09-19T04:04:39.323270+0000","last_active":"2024-09-19T04:05:43.149666+0000","last_peered":"2024-09-19T04:05:43.149666+0000","last_clean":"2024-09-19T04:05:43.149666+0000","last_became_active":"2024-09-19T04:04:39.322890+0000","last_became_peered":"2024-09-19T04:04:39.322890+0000","last_unstale":"2024-09-19T04:05:43.149666+0000","last_undegraded":"2024-09-19T04:05:43.149666+0000","last_fullsized":"2024-09-19T04:05:43.149666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:07.311654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150098+0000","last_change":"2024-09-19T04:04:39.317098+0000","last_active":"2024-09-19T04:05:43.150098+0000","last_peered":"2024-09-19T04:05:43.150098+0000","last_clean":"2024-09-19T04:05:43.150098+0000","last_became_active":"2024-09-19T04:04:39.316792+0000","last_became_peered":"2024-09-19T04:04:39.316792+0000","last_unstale":"2024-09-19T04:05:43.150098+0000","last_undegraded":"2024-09-19T04:05:43.150098+0000","last_fullsized":"2024-09-19T04:05:43.150098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.571149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.465176+0000","last_change":"2024-09-19T04:04:39.326473+0000","last_active":"2024-09-19T04:05:42.465176+0000","last_peered":"2024-09-19T04:05:42.465176+0000","last_clean":"2024-09-19T04:05:42.465176+0000","last_became_active":"2024-09-19T04:04:39.325506+0000","last_became_peered":"2024-09-19T04:04:39.325506+0000","last_unstale":"2024-09-19T04:05:42.465176+0000","last_undegraded":"2024-09-19T04:05:42.465176+0000","last_fullsized":"2024-09-19T04:05:42.465176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:40.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337032+0000","last_change":"2024-09-19T04:04:39.323536+0000","last_active":"2024-09-19T04:05:42.337032+0000","last_peered":"2024-09-19T04:05:42.337032+0000","last_clean":"2024-09-19T04:05:42.337032+0000","last_became_active":"2024-09-19T04:04:39.323224+0000","last_became_peered":"2024-09-19T04:04:39.323224+0000","last_unstale":"2024-09-19T04:05:42.337032+0000","last_undegraded":"2024-09-19T04:05:42.337032+0000","last_fullsized":"2024-09-19T04:05:42.337032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:54.640599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.338305+0000","last_change":"2024-09-19T04:04:39.323802+0000","last_active":"2024-09-19T04:05:42.338305+0000","last_peered":"2024-09-19T04:05:42.338305+0000","last_clean":"2024-09-19T04:05:42.338305+0000","last_became_active":"2024-09-19T04:04:39.323031+0000","last_became_peered":"2024-09-19T04:04:39.323031+0000","last_unstale":"2024-09-19T04:05:42.338305+0000","last_undegraded":"2024-09-19T04:05:42.338305+0000","last_fullsized":"2024-09-19T04:05:42.338305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:12.467182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334632+0000","last_change":"2024-09-19T04:04:39.324870+0000","last_active":"2024-09-19T04:05:42.334632+0000","last_peered":"2024-09-19T04:05:42.334632+0000","last_clean":"2024-09-19T04:05:42.334632+0000","last_became_active":"2024-09-19T04:04:39.324551+0000","last_became_peered":"2024-09-19T04:04:39.324551+0000","last_unstale":"2024-09-19T04:05:42.334632+0000","last_undegraded":"2024-09-19T04:05:42.334632+0000","last_fullsized":"2024-09-19T04:05:42.334632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:34.766437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332266+0000","last_change":"2024-09-19T04:04:39.329086+0000","last_active":"2024-09-19T04:05:42.332266+0000","last_peered":"2024-09-19T04:05:42.332266+0000","last_clean":"2024-09-19T04:05:42.332266+0000","last_became_active":"2024-09-19T04:04:39.328651+0000","last_became_peered":"2024-09-19T04:04:39.328651+0000","last_unstale":"2024-09-19T04:05:42.332266+0000","last_undegraded":"2024-09-19T04:05:42.332266+0000","last_fullsized":"2024-09-19T04:05:42.332266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:12.079162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.822563+0000","last_change":"2024-09-19T04:04:39.325427+0000","last_active":"2024-09-19T04:05:42.822563+0000","last_peered":"2024-09-19T04:05:42.822563+0000","last_clean":"2024-09-19T04:05:42.822563+0000","last_became_active":"2024-09-19T04:04:39.325113+0000","last_became_peered":"2024-09-19T04:04:39.325113+0000","last_unstale":"2024-09-19T04:05:42.822563+0000","last_undegraded":"2024-09-19T04:05:42.822563+0000","last_fullsized":"2024-09-19T04:05:42.822563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:15.844503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.337085+0000","last_change":"2024-09-19T04:04:39.325032+0000","last_active":"2024-09-19T04:05:42.337085+0000","last_peered":"2024-09-19T04:05:42.337085+0000","last_clean":"2024-09-19T04:05:42.337085+0000","last_became_active":"2024-09-19T04:04:39.324618+0000","last_became_peered":"2024-09-19T04:04:39.324618+0000","last_unstale":"2024-09-19T04:05:42.337085+0000","last_undegraded":"2024-09-19T04:05:42.337085+0000","last_fullsized":"2024-09-19T04:05:42.337085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:58.748613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.334110+0000","last_change":"2024-09-19T04:04:39.318336+0000","last_active":"2024-09-19T04:05:42.334110+0000","last_peered":"2024-09-19T04:05:42.334110+0000","last_clean":"2024-09-19T04:05:42.334110+0000","last_became_active":"2024-09-19T04:04:39.317032+0000","last_became_peered":"2024-09-19T04:04:39.317032+0000","last_unstale":"2024-09-19T04:05:42.334110+0000","last_undegraded":"2024-09-19T04:05:42.334110+0000","last_fullsized":"2024-09-19T04:05:42.334110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:04.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.332708+0000","last_change":"2024-09-19T04:04:39.321909+0000","last_active":"2024-09-19T04:05:42.332708+0000","last_peered":"2024-09-19T04:05:42.332708+0000","last_clean":"2024-09-19T04:05:42.332708+0000","last_became_active":"2024-09-19T04:04:39.320522+0000","last_became_peered":"2024-09-19T04:04:39.320522+0000","last_unstale":"2024-09-19T04:05:42.332708+0000","last_undegraded":"2024-09-19T04:05:42.332708+0000","last_fullsized":"2024-09-19T04:05:42.332708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:57.270998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336942+0000","last_change":"2024-09-19T04:04:39.324610+0000","last_active":"2024-09-19T04:05:42.336942+0000","last_peered":"2024-09-19T04:05:42.336942+0000","last_clean":"2024-09-19T04:05:42.336942+0000","last_became_active":"2024-09-19T04:04:39.324190+0000","last_became_peered":"2024-09-19T04:04:39.324190+0000","last_unstale":"2024-09-19T04:05:42.336942+0000","last_undegraded":"2024-09-19T04:05:42.336942+0000","last_fullsized":"2024-09-19T04:05:42.336942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:51.418210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.149987+0000","last_change":"2024-09-19T04:04:39.324220+0000","last_active":"2024-09-19T04:05:43.149987+0000","last_peered":"2024-09-19T04:05:43.149987+0000","last_clean":"2024-09-19T04:05:43.149987+0000","last_became_active":"2024-09-19T04:04:39.323890+0000","last_became_peered":"2024-09-19T04:04:39.323890+0000","last_unstale":"2024-09-19T04:05:43.149987+0000","last_undegraded":"2024-09-19T04:05:43.149987+0000","last_fullsized":"2024-09-19T04:05:43.149987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:15.475088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:43.150226+0000","last_change":"2024-09-19T04:04:39.317240+0000","last_active":"2024-09-19T04:05:43.150226+0000","last_peered":"2024-09-19T04:05:43.150226+0000","last_clean":"2024-09-19T04:05:43.150226+0000","last_became_active":"2024-09-19T04:04:39.316740+0000","last_became_peered":"2024-09-19T04:04:39.316740+0000","last_unstale":"2024-09-19T04:05:43.150226+0000","last_undegraded":"2024-09-19T04:05:43.150226+0000","last_fullsized":"2024-09-19T04:05:43.150226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:46.705467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:05:42.336150+0000","last_change":"2024-09-19T04:04:39.319665+0000","last_active":"2024-09-19T04:05:42.336150+0000","last_peered":"2024-09-19T04:05:42.336150+0000","last_clean":"2024-09-19T04:05:42.336150+0000","last_became_active":"2024-09-19T04:04:39.319227+0000","last_became_peered":"2024-09-19T04:04:39.319227+0000","last_unstale":"2024-09-19T04:05:42.336150+0000","last_undegraded":"2024-09-19T04:05:42.336150+0000","last_fullsized":"2024-09-19T04:05:42.336150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:45.581254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68891,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":553,"num_read_kb":351,"num_write":246,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":60960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":206673,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248101,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481825,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":510207,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710757,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1159077,"data_compressed":59360,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874283,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":514211,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":510471,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364853,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1081344,"data_stored":1106992,"data_compressed":59360,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528379,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":506012,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1089536,"data_stored":1102181,"data_compressed":59360,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":571769,"data_compressed":33234,"data_compressed_allocated":45056,"data_compressed_original":122880,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":505327,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":506869,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":569118,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":507075,"data_compressed":23449,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":62776,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:06:18.850 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:06:18.850 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T04:06:18.851 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T04:06:18.851 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T04:06:18.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:18 smithi111 ceph-mon[35365]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:06:18.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:18 smithi196 ceph-mon[35731]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:06:19.109 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/630367583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68102790 msgr2=0x7f1d681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/630367583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68102790 0x7f1d681067e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009900 tx=0x7f1d5c02a410 comp rx=0 tx=0).stop 2024-09-19T04:06:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/630367583 shutdown_connections 2024-09-19T04:06:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/630367583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d68106f10 0x7f1d6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/630367583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68102790 0x7f1d681067e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/630367583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68101de0 0x7f1d681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/630367583 >> 172.21.15.19:0/630367583 conn(0x7f1d680fd8c0 msgr2=0x7f1d680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/630367583 shutdown_connections 2024-09-19T04:06:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.604+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/630367583 wait complete. 2024-09-19T04:06:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6cb6b640 1 Processor -- start 2024-09-19T04:06:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6cb6b640 1 -- start start 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6cb6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68101de0 0x7f1d68198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6cb6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68102790 0x7f1d68198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6cb6b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d68106f10 0x7f1d6819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6cb6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68111f30 con 0x7f1d68101de0 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6cb6b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68111db0 con 0x7f1d68102790 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6cb6b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681120b0 con 0x7f1d68106f10 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68101de0 0x7f1d68198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68101de0 0x7f1d68198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38454/0 (socket says 172.21.15.19:38454) 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d6659c640 1 -- 172.21.15.19:0/2882697894 learned_addr learned my addr 172.21.15.19:0/2882697894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.605+0000 7f1d65d9b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68102790 0x7f1d68198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d66d9d640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d68106f10 0x7f1d6819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d6659c640 1 -- 172.21.15.19:0/2882697894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d68106f10 msgr2=0x7f1d6819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d6659c640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d68106f10 0x7f1d6819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d6659c640 1 -- 172.21.15.19:0/2882697894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68102790 msgr2=0x7f1d68198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d6659c640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68102790 0x7f1d68198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d6659c640 1 -- 172.21.15.19:0/2882697894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6819d4c0 con 0x7f1d68101de0 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d66d9d640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d68106f10 0x7f1d6819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d6659c640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68101de0 0x7f1d68198510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800d950 tx=0x7f1d5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d577fe640 1 -- 172.21.15.19:0/2882697894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58017070 con 0x7f1d68101de0 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6819d7b0 con 0x7f1d68101de0 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d577fe640 1 -- 172.21.15.19:0/2882697894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d58009c30 con 0x7f1d68101de0 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d577fe640 1 -- 172.21.15.19:0/2882697894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800f6b0 con 0x7f1d68101de0 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.606+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d681a5090 con 0x7f1d68101de0 2024-09-19T04:06:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.607+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d681021c0 con 0x7f1d68101de0 2024-09-19T04:06:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.608+0000 7f1d577fe640 1 -- 172.21.15.19:0/2882697894 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1d580109d0 con 0x7f1d68101de0 2024-09-19T04:06:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.609+0000 7f1d577fe640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d500a32d0 0x7f1d500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.610+0000 7f1d65d9b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d500a32d0 0x7f1d500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.610+0000 7f1d577fe640 1 -- 172.21.15.19:0/2882697894 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f1d580cfa70 con 0x7f1d68101de0 2024-09-19T04:06:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.610+0000 7f1d65d9b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d500a32d0 0x7f1d500a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009230 tx=0x7f1d5c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.613+0000 7f1d577fe640 1 -- 172.21.15.19:0/2882697894 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d58009de0 con 0x7f1d68101de0 2024-09-19T04:06:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.826+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1d6810a0f0 con 0x7f1d68101de0 2024-09-19T04:06:19.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:19 smithi019 ceph-mon[30938]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.827+0000 7f1d577fe640 1 -- 172.21.15.19:0/2882697894 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1d5809c290 con 0x7f1d68101de0 2024-09-19T04:06:19.829 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:19.829 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T04:06:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d500a32d0 msgr2=0x7f1d500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d500a32d0 0x7f1d500a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009230 tx=0x7f1d5c0058d0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68101de0 msgr2=0x7f1d68198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68101de0 0x7f1d68198510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800d950 tx=0x7f1d5800de10 comp rx=0 tx=0).stop 2024-09-19T04:06:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 shutdown_connections 2024-09-19T04:06:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d500a32d0 0x7f1d500a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d68106f10 0x7f1d6819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68102790 0x7f1d68198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 --2- 172.21.15.19:0/2882697894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68101de0 0x7f1d68198510 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 >> 172.21.15.19:0/2882697894 conn(0x7f1d680fd8c0 msgr2=0x7f1d680fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 shutdown_connections 2024-09-19T04:06:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:19.829+0000 7f1d6cb6b640 1 -- 172.21.15.19:0/2882697894 wait complete. 2024-09-19T04:06:19.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:19 smithi111 ceph-mon[35365]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:19.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:19 smithi196 ceph-mon[35731]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:20.526 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T04:06:20.527 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:06:20.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:20 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2882697894' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:06:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:20.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:20 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2882697894' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:06:20.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:20 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2882697894' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T04:06:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 -- 172.21.15.19:0/3835750519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 msgr2=0x7f5a90107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3835750519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 0x7f5a90107b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009930 tx=0x7f5a7802a420 comp rx=0 tx=0).stop 2024-09-19T04:06:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 -- 172.21.15.19:0/3835750519 shutdown_connections 2024-09-19T04:06:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3835750519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a90108290 0x7f5a9010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3835750519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 0x7f5a90107b80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3835750519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a90102d90 0x7f5a90103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 -- 172.21.15.19:0/3835750519 >> 172.21.15.19:0/3835750519 conn(0x7f5a900fc7d0 msgr2=0x7f5a900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 -- 172.21.15.19:0/3835750519 shutdown_connections 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.261+0000 7f5a967fd640 1 -- 172.21.15.19:0/3835750519 wait complete. 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a967fd640 1 Processor -- start 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a967fd640 1 -- start start 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a967fd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a90102d90 0x7f5a901985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a967fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 0x7f5a90198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a967fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a90108290 0x7f5a9019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a967fd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a90112050 con 0x7f5a90103740 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a967fd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5a90111ed0 con 0x7f5a90102d90 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a967fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a901121d0 con 0x7f5a90108290 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 0x7f5a90198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 0x7f5a90198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54626/0 (socket says 172.21.15.19:54626) 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.262+0000 7f5a87fff640 1 -- 172.21.15.19:0/3129103527 learned_addr learned my addr 172.21.15.19:0/3129103527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a8ffff640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a90102d90 0x7f5a901985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a87fff640 1 -- 172.21.15.19:0/3129103527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a90108290 msgr2=0x7f5a9019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a94d9a640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a90108290 0x7f5a9019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a87fff640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a90108290 0x7f5a9019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a87fff640 1 -- 172.21.15.19:0/3129103527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a90102d90 msgr2=0x7f5a901985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a87fff640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a90102d90 0x7f5a901985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a87fff640 1 -- 172.21.15.19:0/3129103527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9019d5a0 con 0x7f5a90103740 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a8ffff640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a90102d90 0x7f5a901985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a87fff640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 0x7f5a90198b30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5a78005940 tx=0x7f5a780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a8dffb640 1 -- 172.21.15.19:0/3129103527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7803f030 con 0x7f5a90103740 2024-09-19T04:06:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a8dffb640 1 -- 172.21.15.19:0/3129103527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a78003ce0 con 0x7f5a90103740 2024-09-19T04:06:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9019d830 con 0x7f5a90103740 2024-09-19T04:06:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a8dffb640 1 -- 172.21.15.19:0/3129103527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78035070 con 0x7f5a90103740 2024-09-19T04:06:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.263+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a901a5170 con 0x7f5a90103740 2024-09-19T04:06:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.264+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a90103170 con 0x7f5a90103740 2024-09-19T04:06:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.265+0000 7f5a8dffb640 1 -- 172.21.15.19:0/3129103527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5a78042030 con 0x7f5a90103740 2024-09-19T04:06:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.266+0000 7f5a8dffb640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5a7c0a3350 0x7f5a7c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.266+0000 7f5a8ffff640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5a7c0a3350 0x7f5a7c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.266+0000 7f5a8dffb640 1 -- 172.21.15.19:0/3129103527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5a780f0080 con 0x7f5a90103740 2024-09-19T04:06:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.267+0000 7f5a8ffff640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5a7c0a3350 0x7f5a7c0a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009b40 tx=0x7f5a80005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:21.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.270+0000 7f5a8dffb640 1 -- 172.21.15.19:0/3129103527 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5a780bc8a0 con 0x7f5a90103740 2024-09-19T04:06:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.482+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a90199950 con 0x7f5a90103740 2024-09-19T04:06:21.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.483+0000 7f5a8dffb640 1 -- 172.21.15.19:0/3129103527 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11980 (secure 0 0 0) 0x7f5a90199950 con 0x7f5a90103740 2024-09-19T04:06:21.484 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:21.485 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-19T04:05:23:602377+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:05:23.279052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24454,24452,24448]},"id":1}]} 2024-09-19T04:06:21.485 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-19T04:06:21.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5a7c0a3350 msgr2=0x7f5a7c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:21.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5a7c0a3350 0x7f5a7c0a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009b40 tx=0x7f5a80005e50 comp rx=0 tx=0).stop 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 msgr2=0x7f5a90198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 0x7f5a90198b30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5a78005940 tx=0x7f5a780042f0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 shutdown_connections 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5a7c0a3350 0x7f5a7c0a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a90108290 0x7f5a9019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a90103740 0x7f5a90198b30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 --2- 172.21.15.19:0/3129103527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a90102d90 0x7f5a901985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 >> 172.21.15.19:0/3129103527 conn(0x7f5a900fc7d0 msgr2=0x7f5a90105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 shutdown_connections 2024-09-19T04:06:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:21.485+0000 7f5a967fd640 1 -- 172.21.15.19:0/3129103527 wait complete. 2024-09-19T04:06:21.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:21 smithi019 ceph-mon[30938]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:21.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:21 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3129103527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:21 smithi111 ceph-mon[35365]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:21.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:21 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3129103527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:21.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:21 smithi196 ceph-mon[35731]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:21.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:21 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3129103527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:22.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:06:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:22.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 -- 172.21.15.19:0/2038318257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc103660 msgr2=0x7efccc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 --2- 172.21.15.19:0/2038318257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc103660 0x7efccc10bb50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009900 tx=0x7efcc002a480 comp rx=0 tx=0).stop 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 -- 172.21.15.19:0/2038318257 shutdown_connections 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 --2- 172.21.15.19:0/2038318257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efccc10c170 0x7efccc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 --2- 172.21.15.19:0/2038318257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc103660 0x7efccc10bb50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 --2- 172.21.15.19:0/2038318257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efccc102d40 0x7efccc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 -- 172.21.15.19:0/2038318257 >> 172.21.15.19:0/2038318257 conn(0x7efccc0fc7a0 msgr2=0x7efccc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 -- 172.21.15.19:0/2038318257 shutdown_connections 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.970+0000 7efcd0834640 1 -- 172.21.15.19:0/2038318257 wait complete. 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcd0834640 1 Processor -- start 2024-09-19T04:06:22.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcd0834640 1 -- start start 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcd0834640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efccc102d40 0x7efccc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcd0834640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efccc103660 0x7efccc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcd0834640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc10c170 0x7efccc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcd0834640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efccc111f70 con 0x7efccc10c170 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcd0834640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efccc111df0 con 0x7efccc103660 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcd0834640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1120f0 con 0x7efccc102d40 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efccad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc10c170 0x7efccc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efccad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc10c170 0x7efccc101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54648/0 (socket says 172.21.15.19:54648) 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efccad9d640 1 -- 172.21.15.19:0/4257222499 learned_addr learned my addr 172.21.15.19:0/4257222499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcca59c640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efccc102d40 0x7efccc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcc9d9b640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efccc103660 0x7efccc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efccad9d640 1 -- 172.21.15.19:0/4257222499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efccc102d40 msgr2=0x7efccc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efccad9d640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efccc102d40 0x7efccc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efccad9d640 1 -- 172.21.15.19:0/4257222499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efccc103660 msgr2=0x7efccc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efccad9d640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efccc103660 0x7efccc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efccad9d640 1 -- 172.21.15.19:0/4257222499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc1a6890 con 0x7efccc10c170 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcca59c640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efccc102d40 0x7efccc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.971+0000 7efcc9d9b640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efccc103660 0x7efccc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.972+0000 7efccad9d640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc10c170 0x7efccc101420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efcc400eaa0 tx=0x7efcc400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.972+0000 7efcbb7fe640 1 -- 172.21.15.19:0/4257222499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc400cdb0 con 0x7efccc10c170 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.972+0000 7efcbb7fe640 1 -- 172.21.15.19:0/4257222499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc4005020 con 0x7efccc10c170 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.972+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc1a6b80 con 0x7efccc10c170 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.972+0000 7efcbb7fe640 1 -- 172.21.15.19:0/4257222499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc40116b0 con 0x7efccc10c170 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.972+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc1a70c0 con 0x7efccc10c170 2024-09-19T04:06:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.973+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efccc103120 con 0x7efccc10c170 2024-09-19T04:06:23.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.974+0000 7efcbb7fe640 1 -- 172.21.15.19:0/4257222499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efcc4005560 con 0x7efccc10c170 2024-09-19T04:06:23.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.975+0000 7efcbb7fe640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efcb40a32d0 0x7efcb40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:23.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.975+0000 7efcbb7fe640 1 -- 172.21.15.19:0/4257222499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7efcc40d0070 con 0x7efccc10c170 2024-09-19T04:06:23.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.975+0000 7efcca59c640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efcb40a32d0 0x7efcb40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:23.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.975+0000 7efcca59c640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efcb40a32d0 0x7efcb40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00abe0 tx=0x7efcbc00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:23.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:22.979+0000 7efcbb7fe640 1 -- 172.21.15.19:0/4257222499 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efcc4016030 con 0x7efccc10c170 2024-09-19T04:06:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.191+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efccc1071e0 con 0x7efccc10c170 2024-09-19T04:06:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.192+0000 7efcbb7fe640 1 -- 172.21.15.19:0/4257222499 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11980 (secure 0 0 0) 0x7efcc409c8f0 con 0x7efccc10c170 2024-09-19T04:06:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:23.194 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-19T04:05:23:602377+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:05:23.279052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24454,24452,24448]},"id":1}]} 2024-09-19T04:06:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efcb40a32d0 msgr2=0x7efcb40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efcb40a32d0 0x7efcb40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00abe0 tx=0x7efcbc00a5c0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc10c170 msgr2=0x7efccc101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc10c170 0x7efccc101420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efcc400eaa0 tx=0x7efcc400ef60 comp rx=0 tx=0).stop 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 shutdown_connections 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efcb40a32d0 0x7efcb40a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efccc10c170 0x7efccc101420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efccc103660 0x7efccc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 --2- 172.21.15.19:0/4257222499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efccc102d40 0x7efccc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:23.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 >> 172.21.15.19:0/4257222499 conn(0x7efccc0fc7a0 msgr2=0x7efccc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:23.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 shutdown_connections 2024-09-19T04:06:23.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:23.194+0000 7efcd0834640 1 -- 172.21.15.19:0/4257222499 wait complete. 2024-09-19T04:06:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:23 smithi111 ceph-mon[35365]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:23.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:23 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/4257222499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:23.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:23 smithi019 ceph-mon[30938]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:23.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:23 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4257222499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:23.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:23 smithi196 ceph-mon[35731]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:23.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:23 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/4257222499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:23.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:06:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:24.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/812193993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 msgr2=0x7f4bc410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:24.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/812193993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 0x7f4bc410eaf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009cd0 tx=0x7f4bbc02a8e0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/812193993 shutdown_connections 2024-09-19T04:06:24.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/812193993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 0x7f4bc410eaf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/812193993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc4100ca0 0x7f4bc4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/812193993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc41002f0 0x7f4bc41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/812193993 >> 172.21.15.19:0/812193993 conn(0x7f4bc4077210 msgr2=0x7f4bc40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/812193993 shutdown_connections 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.722+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/812193993 wait complete. 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc9bf7640 1 Processor -- start 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc9bf7640 1 -- start start 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc9bf7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc41002f0 0x7f4bc4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc9bf7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc4100ca0 0x7f4bc41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc9bf7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 0x7f4bc419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc9bf7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc4112000 con 0x7f4bc4106f50 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc9bf7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc4111e80 con 0x7f4bc4100ca0 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc9bf7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc4112180 con 0x7f4bc41002f0 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 0x7f4bc419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 0x7f4bc419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54672/0 (socket says 172.21.15.19:54672) 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc3fff640 1 -- 172.21.15.19:0/3122391870 learned_addr learned my addr 172.21.15.19:0/3122391870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc37fe640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc41002f0 0x7f4bc4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.723+0000 7f4bc2ffd640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc4100ca0 0x7f4bc41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc3fff640 1 -- 172.21.15.19:0/3122391870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc41002f0 msgr2=0x7f4bc4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc3fff640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc41002f0 0x7f4bc4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc3fff640 1 -- 172.21.15.19:0/3122391870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc4100ca0 msgr2=0x7f4bc41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc3fff640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc4100ca0 0x7f4bc41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc3fff640 1 -- 172.21.15.19:0/3122391870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc419d430 con 0x7f4bc4106f50 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc2ffd640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc4100ca0 0x7f4bc41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc3fff640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 0x7f4bc419cd50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006010 tx=0x7f4bbc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc0ff9640 1 -- 172.21.15.19:0/3122391870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc004450 con 0x7f4bc4106f50 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc419d6c0 con 0x7f4bc4106f50 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc0ff9640 1 -- 172.21.15.19:0/3122391870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc003e10 con 0x7f4bc4106f50 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc0ff9640 1 -- 172.21.15.19:0/3122391870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0466e0 con 0x7f4bc4106f50 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.724+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc41a5000 con 0x7f4bc4106f50 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.725+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc41006d0 con 0x7f4bc4106f50 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.726+0000 7f4bc0ff9640 1 -- 172.21.15.19:0/3122391870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4bbc0305a0 con 0x7f4bc4106f50 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.727+0000 7f4bc0ff9640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bac0a32d0 0x7f4bac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.727+0000 7f4bc37fe640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bac0a32d0 0x7f4bac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.727+0000 7f4bc0ff9640 1 -- 172.21.15.19:0/3122391870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f4bbc02e030 con 0x7f4bc4106f50 2024-09-19T04:06:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.728+0000 7f4bc37fe640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bac0a32d0 0x7f4bac0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4005e10 tx=0x7f4bb4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.732+0000 7f4bc0ff9640 1 -- 172.21.15.19:0/3122391870 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4bbc04b060 con 0x7f4bc4106f50 2024-09-19T04:06:24.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.911+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bc410a180 con 0x7f4bc4106f50 2024-09-19T04:06:24.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.912+0000 7f4bc0ff9640 1 -- 172.21.15.19:0/3122391870 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24685 (secure 0 0 0) 0x7f4bbc039380 con 0x7f4bc4106f50 2024-09-19T04:06:24.914 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:24.914 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:05:42.327059+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:06:24.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bac0a32d0 msgr2=0x7f4bac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:24.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bac0a32d0 0x7f4bac0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4005e10 tx=0x7f4bb4005da0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 msgr2=0x7f4bc419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:24.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 0x7f4bc419cd50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006010 tx=0x7f4bbc003c20 comp rx=0 tx=0).stop 2024-09-19T04:06:24.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 shutdown_connections 2024-09-19T04:06:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4bac0a32d0 0x7f4bac0a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4106f50 0x7f4bc419cd50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc4100ca0 0x7f4bc41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 --2- 172.21.15.19:0/3122391870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc41002f0 0x7f4bc4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 >> 172.21.15.19:0/3122391870 conn(0x7f4bc4077210 msgr2=0x7f4bc4105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 shutdown_connections 2024-09-19T04:06:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:24.915+0000 7f4bc9bf7640 1 -- 172.21.15.19:0/3122391870 wait complete. 2024-09-19T04:06:25.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:06:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:25.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:25 smithi111 ceph-mon[35365]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:25.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:25 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3122391870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:06:25.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:25 smithi019 ceph-mon[30938]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:25.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:25 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3122391870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:06:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:25 smithi196 ceph-mon[35731]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:25.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:25 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3122391870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:06:26.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 -- 172.21.15.19:0/3635057443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01036e0 msgr2=0x7fd9c0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/3635057443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01036e0 0x7fd9c0107b90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009930 tx=0x7fd9b402a420 comp rx=0 tx=0).stop 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 -- 172.21.15.19:0/3635057443 shutdown_connections 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/3635057443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c01082a0 0x7fd9c010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/3635057443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01036e0 0x7fd9c0107b90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/3635057443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102d30 0x7fd9c0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 -- 172.21.15.19:0/3635057443 >> 172.21.15.19:0/3635057443 conn(0x7fd9c00fc7d0 msgr2=0x7fd9c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 -- 172.21.15.19:0/3635057443 shutdown_connections 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.339+0000 7fd9c6ece640 1 -- 172.21.15.19:0/3635057443 wait complete. 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c6ece640 1 Processor -- start 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c6ece640 1 -- start start 2024-09-19T04:06:26.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c6ece640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102d30 0x7fd9c01a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c6ece640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c01036e0 0x7fd9c01a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c6ece640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01082a0 0x7fd9c010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c6ece640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01120a0 con 0x7fd9c01082a0 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c6ece640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0111f20 con 0x7fd9c01036e0 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c6ece640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0112220 con 0x7fd9c0102d30 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c546b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01082a0 0x7fd9c010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c546b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01082a0 0x7fd9c010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54686/0 (socket says 172.21.15.19:54686) 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c546b640 1 -- 172.21.15.19:0/2466712821 learned_addr learned my addr 172.21.15.19:0/2466712821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9c4c6a640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102d30 0x7fd9c01a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.340+0000 7fd9bffff640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c01036e0 0x7fd9c01a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9c546b640 1 -- 172.21.15.19:0/2466712821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102d30 msgr2=0x7fd9c01a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9c546b640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102d30 0x7fd9c01a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9c546b640 1 -- 172.21.15.19:0/2466712821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c01036e0 msgr2=0x7fd9c01a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9c546b640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c01036e0 0x7fd9c01a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9c546b640 1 -- 172.21.15.19:0/2466712821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c010e5d0 con 0x7fd9c01082a0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9c546b640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01082a0 0x7fd9c010de50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800eaa0 tx=0x7fd9b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9bdffb640 1 -- 172.21.15.19:0/2466712821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800cdb0 con 0x7fd9c01082a0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c010e8c0 con 0x7fd9c01082a0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9bdffb640 1 -- 172.21.15.19:0/2466712821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b8005020 con 0x7fd9c01082a0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9bdffb640 1 -- 172.21.15.19:0/2466712821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80116b0 con 0x7fd9c01082a0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.341+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c01ab720 con 0x7fd9c01082a0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.342+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c0103110 con 0x7fd9c01082a0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.343+0000 7fd9bdffb640 1 -- 172.21.15.19:0/2466712821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd9b8005560 con 0x7fd9c01082a0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.344+0000 7fd9bdffb640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd9a80a32d0 0x7fd9a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.344+0000 7fd9c4c6a640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd9a80a32d0 0x7fd9a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:26.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.344+0000 7fd9bdffb640 1 -- 172.21.15.19:0/2466712821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fd9b80d0030 con 0x7fd9c01082a0 2024-09-19T04:06:26.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.345+0000 7fd9c4c6a640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd9a80a32d0 0x7fd9a80a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0005fd0 tx=0x7fd9b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:26.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.348+0000 7fd9bdffb640 1 -- 172.21.15.19:0/2466712821 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd9b8016030 con 0x7fd9c01082a0 2024-09-19T04:06:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.561+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9c01069e0 con 0x7fd9c01082a0 2024-09-19T04:06:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.562+0000 7fd9bdffb640 1 -- 172.21.15.19:0/2466712821 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11980 (secure 0 0 0) 0x7fd9b809c850 con 0x7fd9c01082a0 2024-09-19T04:06:26.564 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:26.564 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-19T04:05:23:602377+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:05:23.279052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24454,24452,24448]},"id":1}]} 2024-09-19T04:06:26.564 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-19T04:06:26.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd9a80a32d0 msgr2=0x7fd9a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd9a80a32d0 0x7fd9a80a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0005fd0 tx=0x7fd9b000b390 comp rx=0 tx=0).stop 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01082a0 msgr2=0x7fd9c010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01082a0 0x7fd9c010de50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800eaa0 tx=0x7fd9b800ef60 comp rx=0 tx=0).stop 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 shutdown_connections 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd9a80a32d0 0x7fd9a80a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01082a0 0x7fd9c010de50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c01036e0 0x7fd9c01a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 --2- 172.21.15.19:0/2466712821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102d30 0x7fd9c01a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 >> 172.21.15.19:0/2466712821 conn(0x7fd9c00fc7d0 msgr2=0x7fd9c0103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 shutdown_connections 2024-09-19T04:06:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:26.564+0000 7fd9c6ece640 1 -- 172.21.15.19:0/2466712821 wait complete. 2024-09-19T04:06:26.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:26 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2466712821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:26.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:26 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2466712821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:26.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:26 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2466712821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:27.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:04:40.077653+0000', 'modified': '2024-09-19T04:05:23.279052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24469, 'mds_2': 24454, 'mds_3': 24452, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14574': {'gid': 14574, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/821554910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 821554910}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 821554910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14580': {'gid': 14580, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/733319067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 733319067}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 733319067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6833/3618359356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 3618359356}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 3618359356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/2394579385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 2394579385}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 2394579385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/3717777626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 3717777626}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 3717777626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.196:6835/1584544131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1584544131}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1584544131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.111:6837/1580295791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 1580295791}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 1580295791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.196:6837/1041250110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1041250110}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1041250110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.196:6839/1173261295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1173261295}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1173261295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.111:6839/528342006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 528342006}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 528342006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24476, 24469, 24454, 24452, 24448]} 2024-09-19T04:06:27.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T04:06:27.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T04:06:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:27.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:27 smithi019 ceph-mon[30938]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:27.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:27 smithi111 ceph-mon[35365]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:27.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:27 smithi196 ceph-mon[35731]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:28.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 -- 172.21.15.19:0/3687117997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac106f10 msgr2=0x7f4eac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/3687117997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac106f10 0x7f4eac10ea90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009ca0 tx=0x7f4ea402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 -- 172.21.15.19:0/3687117997 shutdown_connections 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/3687117997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac106f10 0x7f4eac10ea90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/3687117997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eac102790 0x7f4eac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/3687117997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eac101de0 0x7f4eac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 -- 172.21.15.19:0/3687117997 >> 172.21.15.19:0/3687117997 conn(0x7f4eac0fd8c0 msgr2=0x7f4eac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 -- 172.21.15.19:0/3687117997 shutdown_connections 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.099+0000 7f4eb21eb640 1 -- 172.21.15.19:0/3687117997 wait complete. 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eb21eb640 1 Processor -- start 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eb21eb640 1 -- start start 2024-09-19T04:06:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eb21eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac101de0 0x7f4eac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eb21eb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eac102790 0x7f4eac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eb21eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eac106f10 0x7f4eac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eb21eb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac112010 con 0x7f4eac101de0 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eb21eb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac111e90 con 0x7f4eac102790 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eb21eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac112190 con 0x7f4eac106f10 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.100+0000 7f4eab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac101de0 0x7f4eac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac101de0 0x7f4eac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54698/0 (socket says 172.21.15.19:54698) 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eab7fe640 1 -- 172.21.15.19:0/1892306751 learned_addr learned my addr 172.21.15.19:0/1892306751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eac102790 0x7f4eac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eabfff640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eac106f10 0x7f4eac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eab7fe640 1 -- 172.21.15.19:0/1892306751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eac106f10 msgr2=0x7f4eac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eac106f10 0x7f4eac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eab7fe640 1 -- 172.21.15.19:0/1892306751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eac102790 msgr2=0x7f4eac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eac102790 0x7f4eac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eab7fe640 1 -- 172.21.15.19:0/1892306751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac1a6a30 con 0x7f4eac101de0 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eabfff640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eac106f10 0x7f4eac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac101de0 0x7f4eac0744f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00b4a0 tx=0x7f4e9c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4ea97fa640 1 -- 172.21.15.19:0/1892306751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c015020 con 0x7f4eac101de0 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4ea97fa640 1 -- 172.21.15.19:0/1892306751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c011d10 con 0x7f4eac101de0 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eb21eb640 1 -- 172.21.15.19:0/1892306751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac1a6d20 con 0x7f4eac101de0 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4ea97fa640 1 -- 172.21.15.19:0/1892306751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00f2e0 con 0x7f4eac101de0 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.101+0000 7f4eb21eb640 1 -- 172.21.15.19:0/1892306751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac1a7250 con 0x7f4eac101de0 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4ea97fa640 1 -- 172.21.15.19:0/1892306751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4e9c018020 con 0x7f4eac101de0 2024-09-19T04:06:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eb45c4740 1 Processor -- start 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eb45c4740 1 -- start start 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807bddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb80724090 con 0x55fb807be2f0 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fb806cba70 con 0x55fb807bd9d0 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb80720050 con 0x55fb807c8bb0 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54706/0 (socket says 172.21.15.19:54706) 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.104+0000 7f4eab7fe640 1 -- 172.21.15.19:0/3009900049 learned_addr learned my addr 172.21.15.19:0/3009900049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eabfff640 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807bddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eab7fe640 1 -- 172.21.15.19:0/3009900049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 msgr2=0x55fb807cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4ea97fa640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e980a3350 0x7f4e980a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eab7fe640 1 -- 172.21.15.19:0/3009900049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 msgr2=0x55fb807bddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807bddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eab7fe640 1 -- 172.21.15.19:0/3009900049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb806d8130 con 0x55fb807be2f0 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eabfff640 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807bddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eabfff640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e980a3350 0x7f4e980a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c8670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ccbe0 tx=0x7f4e9c0aaff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924db5c80e7e456d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4ea97fa640 1 -- 172.21.15.19:0/1892306751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f4e9c0cf520 con 0x7f4eac101de0 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4ea1ffb640 1 -- 172.21.15.19:0/3009900049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c09b580 con 0x55fb807be2f0 2024-09-19T04:06:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4ea1ffb640 1 -- 172.21.15.19:0/3009900049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c0a68b0 con 0x55fb807be2f0 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.105+0000 7f4ea1ffb640 1 -- 172.21.15.19:0/3009900049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0092b0 con 0x55fb807be2f0 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3009900049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 msgr2=0x55fb807c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eabfff640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e980a3350 0x7f4e980a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009bf0 tx=0x7f4ea40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c8670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ccbe0 tx=0x7f4e9c0aaff0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3009900049 shutdown_connections 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c8670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3009900049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807bddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3009900049 >> 172.21.15.19:0/3009900049 conn(0x55fb807b9f60 msgr2=0x55fb807ba340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3009900049 shutdown_connections 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3009900049 wait complete. 2024-09-19T04:06:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.106+0000 7f4eb45c4740 1 -- start start 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eb45c4740 1 Processor -- start 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807bd9d0 0x55fb807c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807be2f0 0x55fb807c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb807cb780 con 0x55fb807bd9d0 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fb806def20 con 0x55fb807c8bb0 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb807cb900 con 0x55fb807be2f0 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eb45c4740 10 client.0 _ll_register_callbacks cb 0x55fb806a8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807bd9d0 0x55fb807c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807bd9d0 0x55fb807c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54716/0 (socket says 172.21.15.19:54716) 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eab7fe640 1 -- 172.21.15.19:0/22020402 learned_addr learned my addr 172.21.15.19:0/22020402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807be2f0 0x55fb807c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eabfff640 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eab7fe640 1 -- 172.21.15.19:0/22020402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807be2f0 msgr2=0x55fb807c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807be2f0 0x55fb807c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eab7fe640 1 -- 172.21.15.19:0/22020402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 msgr2=0x55fb807c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.107+0000 7f4eab7fe640 1 -- 172.21.15.19:0/22020402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb804952b0 con 0x55fb807bd9d0 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807bd9d0 0x55fb807c82b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c09b4b0 tx=0x7f4e9c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e532775513ae939 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4e8affd640 1 -- 172.21.15.19:0/22020402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0a6e80 con 0x55fb807bd9d0 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4e8affd640 1 -- 172.21.15.19:0/22020402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f4e9c0d2d00 con 0x55fb807bd9d0 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 1 -- 172.21.15.19:0/22020402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fb80731600 con 0x55fb807bd9d0 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4e8affd640 1 -- 172.21.15.19:0/22020402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e9c09b110 con 0x55fb807bd9d0 2024-09-19T04:06:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 10 client.15033 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 1 -- 172.21.15.19:0/22020402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb807cb780 con 0x55fb807bd9d0 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4e8affd640 1 -- 172.21.15.19:0/22020402 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fb80731600 con 0x55fb807bd9d0 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4e8affd640 1 -- 172.21.15.19:0/22020402 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4e9c0ccd50 con 0x55fb807bd9d0 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 10 client.15033 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 10 client.15033 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 20 client.15033 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x55fb807d90a0 0x55fb807da510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 4 client.15033 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.108+0000 7f4eb45c4740 1 -- 172.21.15.19:0/22020402 --> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb8072bb10 con 0x55fb807d90a0 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.109+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x55fb807d90a0 0x55fb807da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.109+0000 7f4e8affd640 10 client.15033 ms_handle_connect on v2:172.21.15.196:6834/1584544131 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.109+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x55fb807d90a0 0x55fb807da510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d48a0830123cb55 server_cookie=c91b33398d1699e5 in_seq=0 out_seq=0 2024-09-19T04:06:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4e8affd640 1 -- 172.21.15.19:0/22020402 <== mds.4 v2:172.21.15.196:6834/1584544131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb8072bb10 con 0x55fb807d90a0 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4e8affd640 10 client.15033 handle_command_reply: tid=0 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 client.15033 shutdown 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 -- 172.21.15.19:0/22020402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807bd9d0 msgr2=0x55fb807c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807bd9d0 0x55fb807c82b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c09b4b0 tx=0x7f4e9c010ef0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 -- 172.21.15.19:0/22020402 shutdown_connections 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807be2f0 0x55fb807c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x55fb807d90a0 0x55fb807da510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/22020402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807bd9d0 0x55fb807c82b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4e8affd640 0 client.15033 ms_handle_reset on v2:172.21.15.196:6834/1584544131 2024-09-19T04:06:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 -- 172.21.15.19:0/22020402 >> 172.21.15.19:0/22020402 conn(0x55fb807b9f60 msgr2=0x55fb807bfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:28.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 -- 172.21.15.19:0/22020402 shutdown_connections 2024-09-19T04:06:28.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 1 -- 172.21.15.19:0/22020402 wait complete. 2024-09-19T04:06:28.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.110+0000 7f4eb45c4740 20 client.15033 trim_cache size 0 max 16384 2024-09-19T04:06:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.130+0000 7f4eb45c4740 1 Processor -- start 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.130+0000 7f4eb45c4740 1 -- start start 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.130+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.130+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.130+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807c4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb807d8850 con 0x55fb807be2f0 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fb807d8f20 con 0x55fb807bd9d0 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb807d95f0 con 0x55fb807c8bb0 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54720/0 (socket says 172.21.15.19:54720) 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 -- 172.21.15.19:0/268875622 learned_addr learned my addr 172.21.15.19:0/268875622 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4eabfff640 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 -- 172.21.15.19:0/268875622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 msgr2=0x55fb807c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 -- 172.21.15.19:0/268875622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 msgr2=0x55fb807c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 -- 172.21.15.19:0/268875622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb807b9d90 con 0x55fb807be2f0 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4eabfff640 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c42e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e94006e10 tx=0x7f4e94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d186660ed71694bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea17fa640 1 -- 172.21.15.19:0/268875622 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94016ac0 con 0x55fb807be2f0 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea17fa640 1 -- 172.21.15.19:0/268875622 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e94020030 con 0x55fb807be2f0 2024-09-19T04:06:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.131+0000 7f4ea17fa640 1 -- 172.21.15.19:0/268875622 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9401c4b0 con 0x55fb807be2f0 2024-09-19T04:06:28.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 -- 172.21.15.19:0/268875622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 msgr2=0x55fb807c42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c42e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e94006e10 tx=0x7f4e94008040 comp rx=0 tx=0).stop 2024-09-19T04:06:28.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 -- 172.21.15.19:0/268875622 shutdown_connections 2024-09-19T04:06:28.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807c8bb0 0x55fb807c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c42e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/268875622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807bd9d0 0x55fb807c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 -- 172.21.15.19:0/268875622 >> 172.21.15.19:0/268875622 conn(0x55fb807b9f60 msgr2=0x55fb807d80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 -- 172.21.15.19:0/268875622 shutdown_connections 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.132+0000 7f4eb45c4740 1 -- 172.21.15.19:0/268875622 wait complete. 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 1 -- start start 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 1 Processor -- start 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807bd9d0 0x55fb807e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb807da390 con 0x55fb807be2f0 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fb807d9cc0 con 0x55fb807c8bb0 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb807daa60 con 0x55fb807bd9d0 2024-09-19T04:06:28.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eb45c4740 10 client.0 _ll_register_callbacks cb 0x7f4e9c0d2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4ea3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4ea3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54724/0 (socket says 172.21.15.19:54724) 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4ea3fff640 1 -- 172.21.15.19:0/3131177265 learned_addr learned my addr 172.21.15.19:0/3131177265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807bd9d0 0x55fb807e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4ea3fff640 1 -- 172.21.15.19:0/3131177265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807bd9d0 msgr2=0x55fb807e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.133+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807bd9d0 0x55fb807e2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4ea3fff640 1 -- 172.21.15.19:0/3131177265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 msgr2=0x55fb807ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4eabfff640 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4ea3fff640 1 -- 172.21.15.19:0/3131177265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb807ca9e0 con 0x55fb807be2f0 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4eabfff640 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4ea3fff640 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c9c70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009910 tx=0x7f4e94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264566609373a3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4e8affd640 1 -- 172.21.15.19:0/3131177265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940059c0 con 0x55fb807be2f0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3131177265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e94020030 con 0x55fb807be2f0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4e8affd640 1 -- 172.21.15.19:0/3131177265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f4e94005b60 con 0x55fb807be2f0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4e8affd640 1 -- 172.21.15.19:0/3131177265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e94005d60 con 0x55fb807be2f0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4eb45c4740 10 client.15039 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3131177265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb807df740 con 0x55fb807be2f0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.134+0000 7f4e8affd640 1 -- 172.21.15.19:0/3131177265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e94020030 con 0x55fb807be2f0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4e8affd640 1 -- 172.21.15.19:0/3131177265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4e940064b0 con 0x55fb807be2f0 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4eb45c4740 10 client.15039 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4eb45c4740 10 client.15039 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:06:28.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4eb45c4740 20 client.15039 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x55fb80847dd0 0x55fb80846ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4eb45c4740 4 client.15039 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3131177265 --> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] -- command(tid 0: {"prefix": "status"}) -- 0x55fb806a80e0 con 0x55fb80847dd0 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x55fb80847dd0 0x55fb80846ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4e8affd640 10 client.15039 ms_handle_connect on v2:172.21.15.196:6834/1584544131 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.135+0000 7f4eab7fe640 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x55fb80847dd0 0x55fb80846ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f439c73229ab59c server_cookie=681ee8dbea1996a4 in_seq=0 out_seq=0 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4e8affd640 1 -- 172.21.15.19:0/3131177265 <== mds.4 v2:172.21.15.196:6834/1584544131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fb806a80e0 con 0x55fb80847dd0 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4e8affd640 10 client.15039 handle_command_reply: tid=0 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 client.15039 shutdown 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3131177265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 msgr2=0x55fb807c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c9c70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009910 tx=0x7f4e94000bc0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3131177265 shutdown_connections 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb807c8bb0 0x55fb807ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x55fb80847dd0 0x55fb80846ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb807be2f0 0x55fb807c9c70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 --2- 172.21.15.19:0/3131177265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb807bd9d0 0x55fb807e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4e8affd640 0 client.15039 ms_handle_reset on v2:172.21.15.196:6834/1584544131 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3131177265 >> 172.21.15.19:0/3131177265 conn(0x55fb807b9f60 msgr2=0x55fb807c3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3131177265 shutdown_connections 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 1 -- 172.21.15.19:0/3131177265 wait complete. 2024-09-19T04:06:28.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.136+0000 7f4eb45c4740 20 client.15039 trim_cache size 0 max 16384 2024-09-19T04:06:28.144 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:28.144 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:28.144 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-09-19T04:06:28.144 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24448, 2024-09-19T04:06:28.144 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:28.144 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:28.144 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:28.144 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 65.615734380000006, 2024-09-19T04:06:28.145 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:28.145 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 91, 2024-09-19T04:06:28.145 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:28.145 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 134.75633086900001 2024-09-19T04:06:28.145 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 -- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e980a3350 msgr2=0x7f4e980a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e980a3350 0x7f4e980a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009bf0 tx=0x7f4ea40095c0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 -- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac101de0 msgr2=0x7f4eac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac101de0 0x7f4eac0744f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00b4a0 tx=0x7f4e9c00b8a0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 -- 172.21.15.19:0/1892306751 shutdown_connections 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e980a3350 0x7f4e980a5810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eac106f10 0x7f4eac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eac102790 0x7f4eac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 --2- 172.21.15.19:0/1892306751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac101de0 0x7f4eac0744f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 -- 172.21.15.19:0/1892306751 >> 172.21.15.19:0/1892306751 conn(0x7f4eac0fd8c0 msgr2=0x7f4eac0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:28.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 -- 172.21.15.19:0/1892306751 shutdown_connections 2024-09-19T04:06:28.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:28.137+0000 7f4eb21eb640 1 -- 172.21.15.19:0/1892306751 wait complete. 2024-09-19T04:06:28.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T04:06:29.075 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 -- 172.21.15.19:0/3147885342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c1008d0 msgr2=0x7f628c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 --2- 172.21.15.19:0/3147885342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c1008d0 0x7f628c100cb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6288009900 tx=0x7f628802a410 comp rx=0 tx=0).stop 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 -- 172.21.15.19:0/3147885342 shutdown_connections 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 --2- 172.21.15.19:0/3147885342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c10e480 0x7f628c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 --2- 172.21.15.19:0/3147885342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f628c101280 0x7f628c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 --2- 172.21.15.19:0/3147885342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c1008d0 0x7f628c100cb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 -- 172.21.15.19:0/3147885342 >> 172.21.15.19:0/3147885342 conn(0x7f628c0fc7a0 msgr2=0x7f628c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 -- 172.21.15.19:0/3147885342 shutdown_connections 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.592+0000 7f6293a97640 1 -- 172.21.15.19:0/3147885342 wait complete. 2024-09-19T04:06:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6293a97640 1 Processor -- start 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6293a97640 1 -- start start 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6293a97640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c1008d0 0x7f628c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6293a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c101280 0x7f628c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6293a97640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f628c10e480 0x7f628c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6293a97640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f628c114200 con 0x7f628c101280 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6293a97640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f628c114080 con 0x7f628c10e480 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6293a97640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f628c114380 con 0x7f628c1008d0 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6291032640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c101280 0x7f628c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6291032640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c101280 0x7f628c19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54750/0 (socket says 172.21.15.19:54750) 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6291032640 1 -- 172.21.15.19:0/2260250327 learned_addr learned my addr 172.21.15.19:0/2260250327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.593+0000 7f6292034640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f628c10e480 0x7f628c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6291833640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c1008d0 0x7f628c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6291032640 1 -- 172.21.15.19:0/2260250327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c1008d0 msgr2=0x7f628c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6291032640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c1008d0 0x7f628c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6291032640 1 -- 172.21.15.19:0/2260250327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f628c10e480 msgr2=0x7f628c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6291032640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f628c10e480 0x7f628c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6291032640 1 -- 172.21.15.19:0/2260250327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c19f6f0 con 0x7f628c101280 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6292034640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f628c10e480 0x7f628c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6291032640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c101280 0x7f628c19ac80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f627c00b4a0 tx=0x7f627c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6282ffd640 1 -- 172.21.15.19:0/2260250327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c014070 con 0x7f628c101280 2024-09-19T04:06:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6282ffd640 1 -- 172.21.15.19:0/2260250327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627c010af0 con 0x7f628c101280 2024-09-19T04:06:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6293a97640 1 -- 172.21.15.19:0/2260250327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628c19f9e0 con 0x7f628c101280 2024-09-19T04:06:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6282ffd640 1 -- 172.21.15.19:0/2260250327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c017d10 con 0x7f628c101280 2024-09-19T04:06:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.594+0000 7f6293a97640 1 -- 172.21.15.19:0/2260250327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628c1a72c0 con 0x7f628c101280 2024-09-19T04:06:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.596+0000 7f6282ffd640 1 -- 172.21.15.19:0/2260250327 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f627c017070 con 0x7f628c101280 2024-09-19T04:06:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6295e70740 1 Processor -- start 2024-09-19T04:06:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6295e70740 1 -- start start 2024-09-19T04:06:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6295e70740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d69d0 0x55d10b4d6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6295e70740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4e1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6295e70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4e3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6295e70740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d10b43d090 con 0x55d10b4d72f0 2024-09-19T04:06:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6295e70740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d10b3e4a70 con 0x55d10b4d69d0 2024-09-19T04:06:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6295e70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d10b439050 con 0x55d10b4e1bb0 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6282ffd640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62780a3350 0x7f62780a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6291833640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4e1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6291833640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4e1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54762/0 (socket says 172.21.15.19:54762) 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6291833640 1 -- 172.21.15.19:0/44199604 learned_addr learned my addr 172.21.15.19:0/44199604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6292034640 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d69d0 0x55d10b4d6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.597+0000 7f6292034640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62780a3350 0x7f62780a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6282ffd640 1 -- 172.21.15.19:0/2260250327 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f627c0cf0d0 con 0x7f628c101280 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6291032640 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4e3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6291833640 1 -- 172.21.15.19:0/44199604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 msgr2=0x55d10b4e3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6291833640 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4e3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6291833640 1 -- 172.21.15.19:0/44199604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d69d0 msgr2=0x55d10b4d6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6291833640 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d69d0 0x55d10b4d6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6291833640 1 -- 172.21.15.19:0/44199604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d10b3f1130 con 0x55d10b4d72f0 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6291833640 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4e1670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6288005ab0 tx=0x7f6288031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b059904ecec8a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6292034640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62780a3350 0x7f62780a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6284007310 tx=0x7f62840072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6277fff640 1 -- 172.21.15.19:0/44199604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628803f070 con 0x55d10b4d72f0 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6277fff640 1 -- 172.21.15.19:0/44199604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6288031a20 con 0x55d10b4d72f0 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.598+0000 7f6277fff640 1 -- 172.21.15.19:0/44199604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288035070 con 0x55d10b4d72f0 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 -- 172.21.15.19:0/44199604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 msgr2=0x55d10b4e1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4e1670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6288005ab0 tx=0x7f6288031630 comp rx=0 tx=0).stop 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 -- 172.21.15.19:0/44199604 shutdown_connections 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4e3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4e1670 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 --2- 172.21.15.19:0/44199604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d69d0 0x55d10b4d6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 -- 172.21.15.19:0/44199604 >> 172.21.15.19:0/44199604 conn(0x55d10b4d2f60 msgr2=0x55d10b4d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 -- 172.21.15.19:0/44199604 shutdown_connections 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.599+0000 7f6295e70740 1 -- 172.21.15.19:0/44199604 wait complete. 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 1 -- start start 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 1 Processor -- start 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4d69d0 0x55d10b4e1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d10b4e4780 con 0x55d10b4d72f0 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d10b3f7f20 con 0x55d10b4e1bb0 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d10b4e4900 con 0x55d10b4d69d0 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6295e70740 10 client.0 _ll_register_callbacks cb 0x55d10b3c1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6291032640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6291032640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4de7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54770/0 (socket says 172.21.15.19:54770) 2024-09-19T04:06:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6291032640 1 -- 172.21.15.19:0/1607150231 learned_addr learned my addr 172.21.15.19:0/1607150231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.600+0000 7f6292034640 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6291833640 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4d69d0 0x55d10b4e1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6291032640 1 -- 172.21.15.19:0/1607150231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4d69d0 msgr2=0x55d10b4e1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6291032640 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4d69d0 0x55d10b4e1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6291032640 1 -- 172.21.15.19:0/1607150231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4e1bb0 msgr2=0x55d10b4decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6291032640 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6291032640 1 -- 172.21.15.19:0/1607150231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d10b1ae2b0 con 0x55d10b4d72f0 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6291032640 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4de7b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f627c00b470 tx=0x7f627c06d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bbf23bfd4a0b93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f62757fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f62757fa640 1 -- 172.21.15.19:0/1607150231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c01e040 con 0x55d10b4d72f0 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f62757fa640 1 -- 172.21.15.19:0/1607150231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f627c097970 con 0x55d10b4d72f0 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6295e70740 1 -- 172.21.15.19:0/1607150231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6288031a20 con 0x55d10b4d72f0 2024-09-19T04:06:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f62757fa640 1 -- 172.21.15.19:0/1607150231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f627c09b300 con 0x55d10b4d72f0 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6295e70740 10 client.15051 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f6295e70740 1 -- 172.21.15.19:0/1607150231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d10b4f0100 con 0x55d10b4d72f0 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.601+0000 7f62757fa640 1 -- 172.21.15.19:0/1607150231 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6288031a20 con 0x55d10b4d72f0 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f62757fa640 1 -- 172.21.15.19:0/1607150231 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f627c0b4140 con 0x55d10b4d72f0 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f6295e70740 10 client.15051 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f6295e70740 10 client.15051 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f6295e70740 20 client.15051 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f6295e70740 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55d10b4f2770 0x55d10b4f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f6295e70740 4 client.15051 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f6295e70740 1 -- 172.21.15.19:0/1607150231 --> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d10b44a600 con 0x55d10b4f2770 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f6291833640 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55d10b4f2770 0x55d10b4f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f6291833640 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55d10b4f2770 0x55d10b4f3be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da71a2884b084584 server_cookie=353b8fb0ddbfd611 in_seq=0 out_seq=0 2024-09-19T04:06:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.602+0000 7f62757fa640 10 client.15051 ms_handle_connect on v2:172.21.15.111:6836/1580295791 2024-09-19T04:06:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.603+0000 7f62757fa640 1 -- 172.21.15.19:0/1607150231 <== mds.3 v2:172.21.15.111:6836/1580295791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d10b44a600 con 0x55d10b4f2770 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.603+0000 7f62757fa640 10 client.15051 handle_command_reply: tid=0 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.603+0000 7f6295e70740 1 client.15051 shutdown 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.603+0000 7f6295e70740 1 -- 172.21.15.19:0/1607150231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 msgr2=0x55d10b4de7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.603+0000 7f6295e70740 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4de7b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f627c00b470 tx=0x7f627c06d0d0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 1 -- 172.21.15.19:0/1607150231 shutdown_connections 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55d10b4f2770 0x55d10b4f3be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d72f0 0x55d10b4de7b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 1 --2- 172.21.15.19:0/1607150231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4d69d0 0x55d10b4e1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f62757fa640 0 client.15051 ms_handle_reset on v2:172.21.15.111:6836/1580295791 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 1 -- 172.21.15.19:0/1607150231 >> 172.21.15.19:0/1607150231 conn(0x55d10b4d2f60 msgr2=0x55d10b4d8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 1 -- 172.21.15.19:0/1607150231 shutdown_connections 2024-09-19T04:06:29.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 1 -- 172.21.15.19:0/1607150231 wait complete. 2024-09-19T04:06:29.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.604+0000 7f6295e70740 20 client.15051 trim_cache size 0 max 16384 2024-09-19T04:06:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6295e70740 1 Processor -- start 2024-09-19T04:06:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6295e70740 1 -- start start 2024-09-19T04:06:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6295e70740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6295e70740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4de770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6295e70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6295e70740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d10b4f1f20 con 0x55d10b4d69d0 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6295e70740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d10b4f25f0 con 0x55d10b4d72f0 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6295e70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d10b4f2cc0 con 0x55d10b4e1bb0 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.622+0000 7f6291833640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291833640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54786/0 (socket says 172.21.15.19:54786) 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291833640 1 -- 172.21.15.19:0/3814998787 learned_addr learned my addr 172.21.15.19:0/3814998787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291032640 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291833640 1 -- 172.21.15.19:0/3814998787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 msgr2=0x55d10b4decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6292034640 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291833640 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291833640 1 -- 172.21.15.19:0/3814998787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 msgr2=0x55d10b4de770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291833640 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4de770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291833640 1 -- 172.21.15.19:0/3814998787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d10b4d3be0 con 0x55d10b4d69d0 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6292034640 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291032640 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4de770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f6291833640 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f3be0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6288031550 tx=0x7f6288005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8154fb735f3ed872 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f62777fe640 1 -- 172.21.15.19:0/3814998787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288052070 con 0x55d10b4d69d0 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f62777fe640 1 -- 172.21.15.19:0/3814998787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6288004300 con 0x55d10b4d69d0 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.623+0000 7f62777fe640 1 -- 172.21.15.19:0/3814998787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288035040 con 0x55d10b4d69d0 2024-09-19T04:06:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 -- 172.21.15.19:0/3814998787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 msgr2=0x55d10b4f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f3be0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6288031550 tx=0x7f6288005690 comp rx=0 tx=0).stop 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 -- 172.21.15.19:0/3814998787 shutdown_connections 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b4decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4de770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 --2- 172.21.15.19:0/3814998787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f3be0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 -- 172.21.15.19:0/3814998787 >> 172.21.15.19:0/3814998787 conn(0x55d10b4d2f60 msgr2=0x55d10b4cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 -- 172.21.15.19:0/3814998787 shutdown_connections 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 -- 172.21.15.19:0/3814998787 wait complete. 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 -- start start 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.624+0000 7f6295e70740 1 Processor -- start 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6295e70740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6295e70740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4f7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6295e70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b55ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6295e70740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d10b4f3a60 con 0x55d10b4d69d0 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6295e70740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d10b4f3390 con 0x55d10b4d72f0 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6295e70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d10b4f4130 con 0x55d10b4e1bb0 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291833640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6295e70740 10 client.0 _ll_register_callbacks cb 0x7f627c009310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291833640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54798/0 (socket says 172.21.15.19:54798) 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291833640 1 -- 172.21.15.19:0/1293589716 learned_addr learned my addr 172.21.15.19:0/1293589716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291032640 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6292034640 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b55ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291833640 1 -- 172.21.15.19:0/1293589716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 msgr2=0x55d10b55ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291833640 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b55ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291833640 1 -- 172.21.15.19:0/1293589716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 msgr2=0x55d10b4f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291833640 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6291833640 1 -- 172.21.15.19:0/1293589716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d10b55b3b0 con 0x55d10b4d69d0 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.625+0000 7f6292034640 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b55ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6291833640 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f71d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6288031670 tx=0x7f6288000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9fe19df4ac7a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f62757fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f62757fa640 1 -- 172.21.15.19:0/1293589716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288052070 con 0x55d10b4d69d0 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f62757fa640 1 -- 172.21.15.19:0/1293589716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f628804c410 con 0x55d10b4d69d0 2024-09-19T04:06:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 1 -- 172.21.15.19:0/1293589716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6288004300 con 0x55d10b4d69d0 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f62757fa640 1 -- 172.21.15.19:0/1293589716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f628805e050 con 0x55d10b4d69d0 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 10 client.15057 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 1 -- 172.21.15.19:0/1293589716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d10b55c4c0 con 0x55d10b4d69d0 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f62757fa640 1 -- 172.21.15.19:0/1293589716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6288004300 con 0x55d10b4d69d0 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f62757fa640 1 -- 172.21.15.19:0/1293589716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f628803bb00 con 0x55d10b4d69d0 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 10 client.15057 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 10 client.15057 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 20 client.15057 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55d10b560940 0x55d10b55f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 4 client.15057 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.626+0000 7f6295e70740 1 -- 172.21.15.19:0/1293589716 --> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] -- command(tid 0: {"prefix": "status"}) -- 0x55d10b444b10 con 0x55d10b560940 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.627+0000 7f6291032640 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55d10b560940 0x55d10b55f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.627+0000 7f62757fa640 10 client.15057 ms_handle_connect on v2:172.21.15.111:6836/1580295791 2024-09-19T04:06:29.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.627+0000 7f6291032640 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55d10b560940 0x55d10b55f650 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2cb71c4320652588 server_cookie=847f74ad058e9476 in_seq=0 out_seq=0 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.627+0000 7f62757fa640 1 -- 172.21.15.19:0/1293589716 <== mds.3 v2:172.21.15.111:6836/1580295791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d10b444b10 con 0x55d10b560940 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.627+0000 7f62757fa640 10 client.15057 handle_command_reply: tid=0 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.627+0000 7f6295e70740 1 client.15057 shutdown 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 -- 172.21.15.19:0/1293589716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 msgr2=0x55d10b4f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f71d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6288031670 tx=0x7f6288000bc0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 -- 172.21.15.19:0/1293589716 shutdown_connections 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55d10b560940 0x55d10b55f650 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d10b4e1bb0 0x55d10b55ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d10b4d72f0 0x55d10b4f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 --2- 172.21.15.19:0/1293589716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d10b4d69d0 0x55d10b4f71d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f62757fa640 0 client.15057 ms_handle_reset on v2:172.21.15.111:6836/1580295791 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 -- 172.21.15.19:0/1293589716 >> 172.21.15.19:0/1293589716 conn(0x55d10b4d2f60 msgr2=0x55d10b4de770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 -- 172.21.15.19:0/1293589716 shutdown_connections 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 1 -- 172.21.15.19:0/1293589716 wait complete. 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6295e70740 20 client.15057 trim_cache size 0 max 16384 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24452, 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:29.637 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:29.638 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:29.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 68.122943401000001, 2024-09-19T04:06:29.638 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:29.638 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 91, 2024-09-19T04:06:29.638 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:29.638 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 130.857142592 2024-09-19T04:06:29.638 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6293a97640 1 -- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62780a3350 msgr2=0x7f62780a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6293a97640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62780a3350 0x7f62780a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6284007310 tx=0x7f62840072a0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6293a97640 1 -- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c101280 msgr2=0x7f628c19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.628+0000 7f6293a97640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c101280 0x7f628c19ac80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f627c00b4a0 tx=0x7f627c00b960 comp rx=0 tx=0).stop 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.629+0000 7f6293a97640 1 -- 172.21.15.19:0/2260250327 shutdown_connections 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.629+0000 7f6293a97640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f62780a3350 0x7f62780a5810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.629+0000 7f6293a97640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f628c10e480 0x7f628c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.629+0000 7f6293a97640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c101280 0x7f628c19ac80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.629+0000 7f6293a97640 1 --2- 172.21.15.19:0/2260250327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c1008d0 0x7f628c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.629+0000 7f6293a97640 1 -- 172.21.15.19:0/2260250327 >> 172.21.15.19:0/2260250327 conn(0x7f628c0fc7a0 msgr2=0x7f628c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.629+0000 7f6293a97640 1 -- 172.21.15.19:0/2260250327 shutdown_connections 2024-09-19T04:06:29.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:29.629+0000 7f6293a97640 1 -- 172.21.15.19:0/2260250327 wait complete. 2024-09-19T04:06:29.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:29 smithi111 ceph-mon[35365]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:29.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:29 smithi019 ceph-mon[30938]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:29.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:29 smithi196 ceph-mon[35731]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:30.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:06:30.662 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 -- 172.21.15.19:0/2146017717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 msgr2=0x7f3700100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 --2- 172.21.15.19:0/2146017717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 0x7f3700100ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009930 tx=0x7f36e802a420 comp rx=0 tx=0).stop 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 -- 172.21.15.19:0/2146017717 shutdown_connections 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 --2- 172.21.15.19:0/2146017717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3700101420 0x7f370010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 --2- 172.21.15.19:0/2146017717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 0x7f3700100ee0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 --2- 172.21.15.19:0/2146017717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3700106a80 0x7f3700106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 -- 172.21.15.19:0/2146017717 >> 172.21.15.19:0/2146017717 conn(0x7f37000fc7a0 msgr2=0x7f37000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 -- 172.21.15.19:0/2146017717 shutdown_connections 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.135+0000 7f3705597640 1 -- 172.21.15.19:0/2146017717 wait complete. 2024-09-19T04:06:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f3705597640 1 Processor -- start 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f3705597640 1 -- start start 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f3705597640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 0x7f370010e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f3705597640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3700101420 0x7f370010ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f3705597640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3700106a80 0x7f370010f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f3705597640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3700111f20 con 0x7f3700100a80 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f3705597640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3700111da0 con 0x7f3700101420 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f3705597640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37001120a0 con 0x7f3700106a80 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f36feffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 0x7f370010e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f36feffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 0x7f370010e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54816/0 (socket says 172.21.15.19:54816) 2024-09-19T04:06:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f36feffd640 1 -- 172.21.15.19:0/4153193464 learned_addr learned my addr 172.21.15.19:0/4153193464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.136+0000 7f36f7fff640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3700101420 0x7f370010ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36ff7fe640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3700106a80 0x7f370010f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36feffd640 1 -- 172.21.15.19:0/4153193464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3700106a80 msgr2=0x7f370010f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36feffd640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3700106a80 0x7f370010f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36feffd640 1 -- 172.21.15.19:0/4153193464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3700101420 msgr2=0x7f370010ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36feffd640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3700101420 0x7f370010ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36feffd640 1 -- 172.21.15.19:0/4153193464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37001a6960 con 0x7f3700100a80 2024-09-19T04:06:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36feffd640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 0x7f370010e670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d950 tx=0x7f36f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36fcff9640 1 -- 172.21.15.19:0/4153193464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0017070 con 0x7f3700100a80 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36fcff9640 1 -- 172.21.15.19:0/4153193464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f0009c30 con 0x7f3700100a80 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f3705597640 1 -- 172.21.15.19:0/4153193464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37001a6c50 con 0x7f3700100a80 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f36fcff9640 1 -- 172.21.15.19:0/4153193464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f000f6b0 con 0x7f3700100a80 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.137+0000 7f3705597640 1 -- 172.21.15.19:0/4153193464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37001a7160 con 0x7f3700100a80 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.139+0000 7f36fcff9640 1 -- 172.21.15.19:0/4153193464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f36f00109d0 con 0x7f3700100a80 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f3707970740 1 Processor -- start 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f3707970740 1 -- start start 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f36fcff9640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f36ec0a32d0 0x7f36ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f3707970740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 0x55d07298bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f3707970740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d072996670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f3707970740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d072996bb0 0x55d072998fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f3707970740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d0728f2090 con 0x55d072996bb0 2024-09-19T04:06:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f3707970740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d072899a70 con 0x55d07298c2f0 2024-09-19T04:06:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f3707970740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d0728ee050 con 0x55d07298b9d0 2024-09-19T04:06:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f36ff7fe640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f36ec0a32d0 0x7f36ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.140+0000 7f36fcff9640 1 -- 172.21.15.19:0/4153193464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f36f00cf9e0 con 0x7f3700100a80 2024-09-19T04:06:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d072996bb0 0x55d072998fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d072996bb0 0x55d072998fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54820/0 (socket says 172.21.15.19:54820) 2024-09-19T04:06:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 -- 172.21.15.19:0/3913615994 learned_addr learned my addr 172.21.15.19:0/3913615994 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36feffd640 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 0x55d07298bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36f7fff640 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d072996670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 -- 172.21.15.19:0/3913615994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 msgr2=0x55d07298bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 0x55d07298bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 -- 172.21.15.19:0/3913615994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 msgr2=0x55d072996670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d072996670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 -- 172.21.15.19:0/3913615994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0728a6130 con 0x55d072996bb0 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36f7fff640 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d072996670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36feffd640 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 0x55d07298bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f36ec0a32d0 0x7f36ec0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f36f8014ca0 tx=0x7f36f8009e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36ff7fe640 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d072996bb0 0x55d072998fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f36f8011c40 tx=0x7f36f801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38004d3e57ca698 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36f57fa640 1 -- 172.21.15.19:0/3913615994 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8022070 con 0x55d072996bb0 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36f57fa640 1 -- 172.21.15.19:0/3913615994 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f8003e50 con 0x55d072996bb0 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.141+0000 7f36f57fa640 1 -- 172.21.15.19:0/3913615994 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8004290 con 0x55d072996bb0 2024-09-19T04:06:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 -- 172.21.15.19:0/3913615994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d072996bb0 msgr2=0x55d072998fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d072996bb0 0x55d072998fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f36f8011c40 tx=0x7f36f801b000 comp rx=0 tx=0).stop 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 -- 172.21.15.19:0/3913615994 shutdown_connections 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d072996bb0 0x55d072998fa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d072996670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 --2- 172.21.15.19:0/3913615994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 0x55d07298bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 -- 172.21.15.19:0/3913615994 >> 172.21.15.19:0/3913615994 conn(0x55d072987f60 msgr2=0x55d072988340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 -- 172.21.15.19:0/3913615994 shutdown_connections 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.142+0000 7f3707970740 1 -- 172.21.15.19:0/3913615994 wait complete. 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 1 -- start start 2024-09-19T04:06:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 1 Processor -- start 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 0x55d072996440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298c2f0 0x55d072993790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 0x55d072993cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d072999780 con 0x55d07298c2f0 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d0728acf20 con 0x55d072996bb0 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d072999900 con 0x55d07298b9d0 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f3707970740 10 client.0 _ll_register_callbacks cb 0x55d072876030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298c2f0 0x55d072993790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298c2f0 0x55d072993790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54822/0 (socket says 172.21.15.19:54822) 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36f7fff640 1 -- 172.21.15.19:0/2156760933 learned_addr learned my addr 172.21.15.19:0/2156760933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36f7fff640 1 -- 172.21.15.19:0/2156760933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 msgr2=0x55d072996440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36f7fff640 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 0x55d072996440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36f7fff640 1 -- 172.21.15.19:0/2156760933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 msgr2=0x55d072993cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36ff7fe640 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 0x55d072993cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36f7fff640 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 0x55d072993cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.143+0000 7f36f7fff640 1 -- 172.21.15.19:0/2156760933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0726632b0 con 0x55d07298c2f0 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f36f7fff640 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298c2f0 0x55d072993790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009930 tx=0x7f36e8031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47df02b2e4d9b110 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f36e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f36e67fc640 1 -- 172.21.15.19:0/2156760933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8035040 con 0x55d07298c2f0 2024-09-19T04:06:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f36e67fc640 1 -- 172.21.15.19:0/2156760933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f36e8004240 con 0x55d07298c2f0 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 1 -- 172.21.15.19:0/2156760933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f8003e50 con 0x55d07298c2f0 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f36e67fc640 1 -- 172.21.15.19:0/2156760933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36e804e050 con 0x55d07298c2f0 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 10 client.15069 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 1 -- 172.21.15.19:0/2156760933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0729a50a0 con 0x55d07298c2f0 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f36e67fc640 1 -- 172.21.15.19:0/2156760933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f8003e50 con 0x55d07298c2f0 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f36e67fc640 1 -- 172.21.15.19:0/2156760933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f36e80456c0 con 0x55d07298c2f0 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 10 client.15069 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 10 client.15069 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 20 client.15069 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55d0729a7720 0x55d0729a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 4 client.15069 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.144+0000 7f3707970740 1 -- 172.21.15.19:0/2156760933 --> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0728ff600 con 0x55d0729a7720 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.145+0000 7f36feffd640 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55d0729a7720 0x55d0729a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.145+0000 7f36e67fc640 10 client.15069 ms_handle_connect on v2:172.21.15.196:6836/1041250110 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.145+0000 7f36feffd640 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55d0729a7720 0x55d0729a8b90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=151b80732cae9331 server_cookie=87bec3275b03b50e in_seq=0 out_seq=0 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f36e67fc640 1 -- 172.21.15.19:0/2156760933 <== mds.2 v2:172.21.15.196:6836/1041250110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0728ff600 con 0x55d0729a7720 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f36e67fc640 10 client.15069 handle_command_reply: tid=0 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 client.15069 shutdown 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 -- 172.21.15.19:0/2156760933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298c2f0 msgr2=0x55d072993790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298c2f0 0x55d072993790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009930 tx=0x7f36e8031860 comp rx=0 tx=0).stop 2024-09-19T04:06:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 -- 172.21.15.19:0/2156760933 shutdown_connections 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55d0729a7720 0x55d0729a8b90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 0x55d072993cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298c2f0 0x55d072993790 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 --2- 172.21.15.19:0/2156760933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298b9d0 0x55d072996440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f36e67fc640 0 client.15069 ms_handle_reset on v2:172.21.15.196:6836/1041250110 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 -- 172.21.15.19:0/2156760933 >> 172.21.15.19:0/2156760933 conn(0x55d072987f60 msgr2=0x55d07298dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 -- 172.21.15.19:0/2156760933 shutdown_connections 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 1 -- 172.21.15.19:0/2156760933 wait complete. 2024-09-19T04:06:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.146+0000 7f3707970740 20 client.15069 trim_cache size 0 max 16384 2024-09-19T04:06:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f3707970740 1 Processor -- start 2024-09-19T04:06:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f3707970740 1 -- start start 2024-09-19T04:06:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f3707970740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f3707970740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298c2f0 0x55d0729909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f3707970740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 0x55d072990f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f3707970740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d0729a6ed0 con 0x55d07298b9d0 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f3707970740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d0729a75a0 con 0x55d072996bb0 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f3707970740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d0729a7c70 con 0x55d07298c2f0 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f36feffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f36feffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729a8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54830/0 (socket says 172.21.15.19:54830) 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f36feffd640 1 -- 172.21.15.19:0/3943422452 learned_addr learned my addr 172.21.15.19:0/3943422452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.165+0000 7f36f7fff640 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298c2f0 0x55d0729909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36ff7fe640 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 0x55d072990f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36feffd640 1 -- 172.21.15.19:0/3943422452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298c2f0 msgr2=0x55d0729909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36feffd640 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298c2f0 0x55d0729909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36feffd640 1 -- 172.21.15.19:0/3943422452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 msgr2=0x55d072990f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36feffd640 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 0x55d072990f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36feffd640 1 -- 172.21.15.19:0/3943422452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d072994550 con 0x55d07298b9d0 2024-09-19T04:06:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36feffd640 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729a8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f36f009b800 tx=0x7f36f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e5c63eb2dde45a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36f4ff9640 1 -- 172.21.15.19:0/3943422452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0020070 con 0x55d07298b9d0 2024-09-19T04:06:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36f4ff9640 1 -- 172.21.15.19:0/3943422452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f0084e50 con 0x55d07298b9d0 2024-09-19T04:06:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.166+0000 7f36f4ff9640 1 -- 172.21.15.19:0/3943422452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0099e00 con 0x55d07298b9d0 2024-09-19T04:06:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 -- 172.21.15.19:0/3943422452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 msgr2=0x55d0729a8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729a8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f36f009b800 tx=0x7f36f0000bc0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 -- 172.21.15.19:0/3943422452 shutdown_connections 2024-09-19T04:06:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d072996bb0 0x55d072990f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d07298c2f0 0x55d0729909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 --2- 172.21.15.19:0/3943422452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729a8b90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 -- 172.21.15.19:0/3943422452 >> 172.21.15.19:0/3943422452 conn(0x55d072987f60 msgr2=0x55d072992880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 -- 172.21.15.19:0/3943422452 shutdown_connections 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 -- 172.21.15.19:0/3943422452 wait complete. 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 -- start start 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.167+0000 7f3707970740 1 Processor -- start 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f3707970740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f3707970740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d0729aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f3707970740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d072996bb0 0x55d0729aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f3707970740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d0729a8a10 con 0x55d07298b9d0 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f3707970740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d0729a8340 con 0x55d07298c2f0 2024-09-19T04:06:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f3707970740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d0729a90e0 con 0x55d072996bb0 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f3707970740 10 client.0 _ll_register_callbacks cb 0x7f36e8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54842/0 (socket says 172.21.15.19:54842) 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 -- 172.21.15.19:0/129940200 learned_addr learned my addr 172.21.15.19:0/129940200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36f7fff640 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d0729aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 -- 172.21.15.19:0/129940200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d072996bb0 msgr2=0x55d0729aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d072996bb0 0x55d0729aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 -- 172.21.15.19:0/129940200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 msgr2=0x55d0729aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d0729aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 -- 172.21.15.19:0/129940200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0729ab100 con 0x55d07298b9d0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36feffd640 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729b1300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f36f00984d0 tx=0x7f36f00a48d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874bc88d638ab7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36e67fc640 1 -- 172.21.15.19:0/129940200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0020070 con 0x55d07298b9d0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f36e67fc640 1 -- 172.21.15.19:0/129940200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f36f00cd240 con 0x55d07298b9d0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.168+0000 7f3707970740 1 -- 172.21.15.19:0/129940200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f0084e50 con 0x55d07298b9d0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f36e67fc640 1 -- 172.21.15.19:0/129940200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36f009c200 con 0x55d07298b9d0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f3707970740 10 client.15075 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f3707970740 1 -- 172.21.15.19:0/129940200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0729afb90 con 0x55d07298b9d0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f36e67fc640 1 -- 172.21.15.19:0/129940200 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f0084e50 con 0x55d07298b9d0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f36e67fc640 1 -- 172.21.15.19:0/129940200 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f36f009af50 con 0x55d07298b9d0 2024-09-19T04:06:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f3707970740 10 client.15075 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f3707970740 10 client.15075 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f3707970740 20 client.15075 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f3707970740 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55d072a16210 0x55d072a14f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f3707970740 4 client.15075 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f3707970740 1 -- 172.21.15.19:0/129940200 --> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] -- command(tid 0: {"prefix": "status"}) -- 0x55d0728f9b10 con 0x55d072a16210 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.169+0000 7f36f7fff640 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55d072a16210 0x55d072a14f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f36f7fff640 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55d072a16210 0x55d072a14f20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=517e52173286d9d6 server_cookie=95c7596bce648266 in_seq=0 out_seq=0 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f36e67fc640 10 client.15075 ms_handle_connect on v2:172.21.15.196:6836/1041250110 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f36e67fc640 1 -- 172.21.15.19:0/129940200 <== mds.2 v2:172.21.15.196:6836/1041250110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d0728f9b10 con 0x55d072a16210 2024-09-19T04:06:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f36e67fc640 10 client.15075 handle_command_reply: tid=0 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 client.15075 shutdown 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 -- 172.21.15.19:0/129940200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 msgr2=0x55d0729b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729b1300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f36f00984d0 tx=0x7f36f00a48d0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 -- 172.21.15.19:0/129940200 shutdown_connections 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55d072a16210 0x55d072a14f20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d072996bb0 0x55d0729aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d07298c2f0 0x55d0729aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 --2- 172.21.15.19:0/129940200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d07298b9d0 0x55d0729b1300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 -- 172.21.15.19:0/129940200 >> 172.21.15.19:0/129940200 conn(0x55d072987f60 msgr2=0x55d07298e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f36e67fc640 0 client.15075 ms_handle_reset on v2:172.21.15.196:6836/1041250110 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 -- 172.21.15.19:0/129940200 shutdown_connections 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 1 -- 172.21.15.19:0/129940200 wait complete. 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.170+0000 7f3707970740 20 client.15075 trim_cache size 0 max 16384 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24454, 2024-09-19T04:06:31.190 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:31.191 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:31.191 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:31.191 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 70.671124563000006, 2024-09-19T04:06:31.191 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:31.191 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 91, 2024-09-19T04:06:31.191 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:31.191 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 135.21092768700001 2024-09-19T04:06:31.191 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 -- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f36ec0a32d0 msgr2=0x7f36ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f36ec0a32d0 0x7f36ec0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f36f8014ca0 tx=0x7f36f8009e70 comp rx=0 tx=0).stop 2024-09-19T04:06:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 -- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 msgr2=0x7f370010e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 0x7f370010e670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d950 tx=0x7f36f000de10 comp rx=0 tx=0).stop 2024-09-19T04:06:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 -- 172.21.15.19:0/4153193464 shutdown_connections 2024-09-19T04:06:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f36ec0a32d0 0x7f36ec0a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3700106a80 0x7f370010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3700101420 0x7f370010ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 --2- 172.21.15.19:0/4153193464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3700100a80 0x7f370010e670 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 -- 172.21.15.19:0/4153193464 >> 172.21.15.19:0/4153193464 conn(0x7f37000fc7a0 msgr2=0x7f37000fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 -- 172.21.15.19:0/4153193464 shutdown_connections 2024-09-19T04:06:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:31.171+0000 7f3705597640 1 -- 172.21.15.19:0/4153193464 wait complete. 2024-09-19T04:06:31.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:31 smithi019 ceph-mon[30938]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:31.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:06:31.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:31 smithi111 ceph-mon[35365]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:31.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:31 smithi196 ceph-mon[35731]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:32.134 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 -- 172.21.15.19:0/1483067440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 msgr2=0x7f558c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 --2- 172.21.15.19:0/1483067440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 0x7f558c10e610 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5584009ca0 tx=0x7f558402a950 comp rx=0 tx=0).stop 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 -- 172.21.15.19:0/1483067440 shutdown_connections 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 --2- 172.21.15.19:0/1483067440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 0x7f558c10e610 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 --2- 172.21.15.19:0/1483067440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f558c103660 0x7f558c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 --2- 172.21.15.19:0/1483067440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f558c102d40 0x7f558c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 -- 172.21.15.19:0/1483067440 >> 172.21.15.19:0/1483067440 conn(0x7f558c0fc7a0 msgr2=0x7f558c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 -- 172.21.15.19:0/1483067440 shutdown_connections 2024-09-19T04:06:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 -- 172.21.15.19:0/1483067440 wait complete. 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.585+0000 7f5591c79640 1 Processor -- start 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f5591c79640 1 -- start start 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f5591c79640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f558c102d40 0x7f558c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f5591c79640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f558c103660 0x7f558c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f5591c79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 0x7f558c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f5591c79640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f558c111f50 con 0x7f558c10c170 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f5591c79640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f558c111dd0 con 0x7f558c103660 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f5591c79640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1120d0 con 0x7f558c102d40 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 0x7f558c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 0x7f558c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60128/0 (socket says 172.21.15.19:60128) 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558bfff640 1 -- 172.21.15.19:0/177291510 learned_addr learned my addr 172.21.15.19:0/177291510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558b7fe640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f558c102d40 0x7f558c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558affd640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f558c103660 0x7f558c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558bfff640 1 -- 172.21.15.19:0/177291510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f558c102d40 msgr2=0x7f558c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558bfff640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f558c102d40 0x7f558c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558bfff640 1 -- 172.21.15.19:0/177291510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f558c103660 msgr2=0x7f558c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558bfff640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f558c103660 0x7f558c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.586+0000 7f558bfff640 1 -- 172.21.15.19:0/177291510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c19d3b0 con 0x7f558c10c170 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.587+0000 7f558bfff640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 0x7f558c19ccd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f558402ae70 tx=0x7f558402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.587+0000 7f5588ff9640 1 -- 172.21.15.19:0/177291510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584034040 con 0x7f558c10c170 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.587+0000 7f5588ff9640 1 -- 172.21.15.19:0/177291510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5584030920 con 0x7f558c10c170 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.587+0000 7f5591c79640 1 -- 172.21.15.19:0/177291510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c19d640 con 0x7f558c10c170 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.587+0000 7f5588ff9640 1 -- 172.21.15.19:0/177291510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584045480 con 0x7f558c10c170 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.587+0000 7f5591c79640 1 -- 172.21.15.19:0/177291510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1a4f80 con 0x7f558c10c170 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.589+0000 7f5588ff9640 1 -- 172.21.15.19:0/177291510 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5584037020 con 0x7f558c10c170 2024-09-19T04:06:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5594052740 1 Processor -- start 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5594052740 1 -- start start 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5594052740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5594052740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5594052740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38e2bb0 0x55cbc38e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5594052740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cbc383e090 con 0x55cbc38d79d0 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5594052740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbc37e5a70 con 0x55cbc38d82f0 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5594052740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbc383a050 con 0x55cbc38e2bb0 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38d7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60130/0 (socket says 172.21.15.19:60130) 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558affd640 1 -- 172.21.15.19:0/571663242 learned_addr learned my addr 172.21.15.19:0/571663242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f5588ff9640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f55740a32d0 0x7f55740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558b7fe640 1 --2- 172.21.15.19:0/571663242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558affd640 1 -- 172.21.15.19:0/571663242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38e2bb0 msgr2=0x55cbc38e4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558affd640 1 --2- 172.21.15.19:0/571663242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38e2bb0 0x55cbc38e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558affd640 1 -- 172.21.15.19:0/571663242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 msgr2=0x55cbc38e2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558affd640 1 --2- 172.21.15.19:0/571663242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558affd640 1 -- 172.21.15.19:0/571663242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc37f2130 con 0x55cbc38d79d0 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.590+0000 7f558bfff640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f55740a32d0 0x7f55740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.591+0000 7f5588ff9640 1 -- 172.21.15.19:0/177291510 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f55840f0be0 con 0x7f558c10c170 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.591+0000 7f558affd640 1 --2- 172.21.15.19:0/571663242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38d7db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5580005940 tx=0x7f55800117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a061f1f17b20f45d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.591+0000 7f5571ffb640 1 -- 172.21.15.19:0/571663242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580005020 con 0x55cbc38d79d0 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.591+0000 7f5571ffb640 1 -- 172.21.15.19:0/571663242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55800047a0 con 0x55cbc38d79d0 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.591+0000 7f5571ffb640 1 -- 172.21.15.19:0/571663242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55800126b0 con 0x55cbc38d79d0 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- 172.21.15.19:0/571663242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 msgr2=0x55cbc38d7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f558bfff640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f55740a32d0 0x7f55740a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55840bcb10 tx=0x7f558408ebe0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 --2- 172.21.15.19:0/571663242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38d7db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5580005940 tx=0x7f55800117b0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- 172.21.15.19:0/571663242 shutdown_connections 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 --2- 172.21.15.19:0/571663242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38e2bb0 0x55cbc38e4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 --2- 172.21.15.19:0/571663242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 --2- 172.21.15.19:0/571663242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38d7db0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- 172.21.15.19:0/571663242 >> 172.21.15.19:0/571663242 conn(0x55cbc38d3f60 msgr2=0x55cbc38d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- 172.21.15.19:0/571663242 shutdown_connections 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- 172.21.15.19:0/571663242 wait complete. 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- start start 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 Processor -- start 2024-09-19T04:06:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d79d0 0x55cbc38e2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d82f0 0x55cbc38df750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cbc38e5780 con 0x55cbc38e2bb0 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbc37f8f20 con 0x55cbc38d79d0 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.592+0000 7f5594052740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbc38e5900 con 0x55cbc38d82f0 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f5594052740 10 client.0 _ll_register_callbacks cb 0x55cbc37c2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38dfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60138/0 (socket says 172.21.15.19:60138) 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 -- 172.21.15.19:0/3957279741 learned_addr learned my addr 172.21.15.19:0/3957279741 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d79d0 0x55cbc38e2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558b7fe640 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d82f0 0x55cbc38df750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 -- 172.21.15.19:0/3957279741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d82f0 msgr2=0x55cbc38df750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d82f0 0x55cbc38df750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 -- 172.21.15.19:0/3957279741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d79d0 msgr2=0x55cbc38e2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d79d0 0x55cbc38e2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 -- 172.21.15.19:0/3957279741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc35af2b0 con 0x55cbc38e2bb0 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f558affd640 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38dfc90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5580004dd0 tx=0x7f558000a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5da46bd434e3301 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f556affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f556affd640 1 -- 172.21.15.19:0/3957279741 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580024040 con 0x55cbc38e2bb0 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f556affd640 1 -- 172.21.15.19:0/3957279741 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5580012650 con 0x55cbc38e2bb0 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.593+0000 7f5594052740 1 -- 172.21.15.19:0/3957279741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55800047a0 con 0x55cbc38e2bb0 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f556affd640 1 -- 172.21.15.19:0/3957279741 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5580012850 con 0x55cbc38e2bb0 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f5594052740 10 client.15087 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f5594052740 1 -- 172.21.15.19:0/3957279741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbc38f1050 con 0x55cbc38e2bb0 2024-09-19T04:06:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f556affd640 1 -- 172.21.15.19:0/3957279741 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55800047a0 con 0x55cbc38e2bb0 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f556affd640 1 -- 172.21.15.19:0/3957279741 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f558002b9c0 con 0x55cbc38e2bb0 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f5594052740 10 client.15087 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f5594052740 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f5594052740 20 client.15087 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f5594052740 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55cbc38f3460 0x55cbc38f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f5594052740 4 client.15087 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.594+0000 7f5594052740 1 -- 172.21.15.19:0/3957279741 --> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbc384b600 con 0x55cbc38f3460 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.595+0000 7f558b7fe640 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55cbc38f3460 0x55cbc38f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.595+0000 7f558b7fe640 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55cbc38f3460 0x55cbc38f48d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdea3f18e4036bef server_cookie=8fcc0eecfca1f68d in_seq=0 out_seq=0 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.595+0000 7f556affd640 10 client.15087 ms_handle_connect on v2:172.21.15.196:6838/1173261295 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f556affd640 1 -- 172.21.15.19:0/3957279741 <== mds.1 v2:172.21.15.196:6838/1173261295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbc384b600 con 0x55cbc38f3460 2024-09-19T04:06:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f556affd640 10 client.15087 handle_command_reply: tid=0 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 client.15087 shutdown 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 -- 172.21.15.19:0/3957279741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 msgr2=0x55cbc38dfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38dfc90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5580004dd0 tx=0x7f558000a810 comp rx=0 tx=0).stop 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 -- 172.21.15.19:0/3957279741 shutdown_connections 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55cbc38f3460 0x55cbc38f48d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38dfc90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d82f0 0x55cbc38df750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f556affd640 0 client.15087 ms_handle_reset on v2:172.21.15.196:6838/1173261295 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 --2- 172.21.15.19:0/3957279741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d79d0 0x55cbc38e2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 -- 172.21.15.19:0/3957279741 >> 172.21.15.19:0/3957279741 conn(0x55cbc38d3f60 msgr2=0x55cbc38d9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 -- 172.21.15.19:0/3957279741 shutdown_connections 2024-09-19T04:06:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 1 -- 172.21.15.19:0/3957279741 wait complete. 2024-09-19T04:06:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.596+0000 7f5594052740 20 client.15087 trim_cache size 0 max 16384 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.614+0000 7f5594052740 1 Processor -- start 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.614+0000 7f5594052740 1 -- start start 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.614+0000 7f5594052740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d79d0 0x55cbc38f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.614+0000 7f5594052740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38f4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.614+0000 7f5594052740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.614+0000 7f5594052740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cbc38f2c10 con 0x55cbc38e2bb0 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.614+0000 7f5594052740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbc38f32e0 con 0x55cbc38d82f0 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.614+0000 7f5594052740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbc38f39b0 con 0x55cbc38d79d0 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60146/0 (socket says 172.21.15.19:60146) 2024-09-19T04:06:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 -- 172.21.15.19:0/2438914640 learned_addr learned my addr 172.21.15.19:0/2438914640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558b7fe640 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d79d0 0x55cbc38f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 -- 172.21.15.19:0/2438914640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d79d0 msgr2=0x55cbc38f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d79d0 0x55cbc38f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 -- 172.21.15.19:0/2438914640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 msgr2=0x55cbc38f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 -- 172.21.15.19:0/2438914640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc38e2770 con 0x55cbc38e2bb0 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f558affd640 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f2d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5580005940 tx=0x7f558000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3241c13e2e198b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f55717fa640 1 -- 172.21.15.19:0/2438914640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580012450 con 0x55cbc38e2bb0 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f55717fa640 1 -- 172.21.15.19:0/2438914640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55800125f0 con 0x55cbc38e2bb0 2024-09-19T04:06:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.615+0000 7f55717fa640 1 -- 172.21.15.19:0/2438914640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580004280 con 0x55cbc38e2bb0 2024-09-19T04:06:32.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 -- 172.21.15.19:0/2438914640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 msgr2=0x55cbc38f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f2d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5580005940 tx=0x7f558000a2d0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 -- 172.21.15.19:0/2438914640 shutdown_connections 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f2d90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 --2- 172.21.15.19:0/2438914640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38d79d0 0x55cbc38f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 -- 172.21.15.19:0/2438914640 >> 172.21.15.19:0/2438914640 conn(0x55cbc38d3f60 msgr2=0x55cbc38f24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 -- 172.21.15.19:0/2438914640 shutdown_connections 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 -- 172.21.15.19:0/2438914640 wait complete. 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.616+0000 7f5594052740 1 -- start start 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f5594052740 1 Processor -- start 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f5594052740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f5594052740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38f8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f5594052740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f5594052740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cbc38f4750 con 0x55cbc38d79d0 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f5594052740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbc38f4080 con 0x55cbc38d82f0 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f5594052740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbc38f4e20 con 0x55cbc38e2bb0 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f5594052740 10 client.0 _ll_register_callbacks cb 0x7f558000ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60160/0 (socket says 172.21.15.19:60160) 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558affd640 1 -- 172.21.15.19:0/2354860794 learned_addr learned my addr 172.21.15.19:0/2354860794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558b7fe640 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558affd640 1 -- 172.21.15.19:0/2354860794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38e2bb0 msgr2=0x55cbc38f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558affd640 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558affd640 1 -- 172.21.15.19:0/2354860794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 msgr2=0x55cbc38f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558affd640 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.617+0000 7f558affd640 1 -- 172.21.15.19:0/2354860794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc395bbb0 con 0x55cbc38d79d0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.618+0000 7f558affd640 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38f7b20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5580001eb0 tx=0x7f558000a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c284814c087a9bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.618+0000 7f556affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.618+0000 7f556affd640 1 -- 172.21.15.19:0/2354860794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580043060 con 0x55cbc38d79d0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 1 -- 172.21.15.19:0/2354860794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55800125f0 con 0x55cbc38d79d0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f556affd640 1 -- 172.21.15.19:0/2354860794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5580004660 con 0x55cbc38d79d0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f556affd640 1 -- 172.21.15.19:0/2354860794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5580012450 con 0x55cbc38d79d0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 10 client.15093 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 1 -- 172.21.15.19:0/2354860794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbc395ceb0 con 0x55cbc38d79d0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f556affd640 1 -- 172.21.15.19:0/2354860794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55800125f0 con 0x55cbc38d79d0 2024-09-19T04:06:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f556affd640 1 -- 172.21.15.19:0/2354860794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5580004280 con 0x55cbc38d79d0 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 10 client.15093 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 20 client.15093 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55cbc3961860 0x55cbc3960570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 4 client.15093 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.619+0000 7f5594052740 1 -- 172.21.15.19:0/2354860794 --> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] -- command(tid 0: {"prefix": "status"}) -- 0x55cbc3845b10 con 0x55cbc3961860 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.620+0000 7f558b7fe640 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55cbc3961860 0x55cbc3960570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.620+0000 7f556affd640 10 client.15093 ms_handle_connect on v2:172.21.15.196:6838/1173261295 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.620+0000 7f558b7fe640 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55cbc3961860 0x55cbc3960570 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a04285505d752ab3 server_cookie=c89522f6cc039cfd in_seq=0 out_seq=0 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.620+0000 7f556affd640 1 -- 172.21.15.19:0/2354860794 <== mds.1 v2:172.21.15.196:6838/1173261295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55cbc3845b10 con 0x55cbc3961860 2024-09-19T04:06:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.620+0000 7f556affd640 10 client.15093 handle_command_reply: tid=0 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.620+0000 7f5594052740 1 client.15093 shutdown 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 -- 172.21.15.19:0/2354860794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 msgr2=0x55cbc38f7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38f7b20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5580001eb0 tx=0x7f558000a9e0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 -- 172.21.15.19:0/2354860794 shutdown_connections 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55cbc3961860 0x55cbc3960570 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc38e2bb0 0x55cbc38f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbc38d82f0 0x55cbc38f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 --2- 172.21.15.19:0/2354860794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbc38d79d0 0x55cbc38f7b20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 -- 172.21.15.19:0/2354860794 >> 172.21.15.19:0/2354860794 conn(0x55cbc38d3f60 msgr2=0x55cbc38ddf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f556affd640 0 client.15093 ms_handle_reset on v2:172.21.15.196:6838/1173261295 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 -- 172.21.15.19:0/2354860794 shutdown_connections 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 1 -- 172.21.15.19:0/2354860794 wait complete. 2024-09-19T04:06:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.621+0000 7f5594052740 20 client.15093 trim_cache size 0 max 16384 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24469, 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 73.130074389000001, 2024-09-19T04:06:32.628 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:32.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 91, 2024-09-19T04:06:32.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:32.629 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 131.28826515 2024-09-19T04:06:32.629 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:32.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 -- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f55740a32d0 msgr2=0x7f55740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f55740a32d0 0x7f55740a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55840bcb10 tx=0x7f558408ebe0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 -- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 msgr2=0x7f558c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 0x7f558c19ccd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f558402ae70 tx=0x7f558402f5b0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 -- 172.21.15.19:0/177291510 shutdown_connections 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f55740a32d0 0x7f55740a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f558c10c170 0x7f558c19ccd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f558c103660 0x7f558c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 --2- 172.21.15.19:0/177291510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f558c102d40 0x7f558c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 -- 172.21.15.19:0/177291510 >> 172.21.15.19:0/177291510 conn(0x7f558c0fc7a0 msgr2=0x7f558c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 -- 172.21.15.19:0/177291510 shutdown_connections 2024-09-19T04:06:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:32.622+0000 7f5591c79640 1 -- 172.21.15.19:0/177291510 wait complete. 2024-09-19T04:06:33.463 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:06:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:33.842 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:33 smithi019 ceph-mon[30938]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:33.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:33 smithi111 ceph-mon[35365]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:33.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:33 smithi196 ceph-mon[35731]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 -- 172.21.15.19:0/3754110635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90101de0 msgr2=0x7f8d901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 --2- 172.21.15.19:0/3754110635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90101de0 0x7f8d901021c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009900 tx=0x7f8d8002a410 comp rx=0 tx=0).stop 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 -- 172.21.15.19:0/3754110635 shutdown_connections 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 --2- 172.21.15.19:0/3754110635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d90106f10 0x7f8d9010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 --2- 172.21.15.19:0/3754110635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d90102790 0x7f8d901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 --2- 172.21.15.19:0/3754110635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90101de0 0x7f8d901021c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 -- 172.21.15.19:0/3754110635 >> 172.21.15.19:0/3754110635 conn(0x7f8d900fd8c0 msgr2=0x7f8d900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 -- 172.21.15.19:0/3754110635 shutdown_connections 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.210+0000 7f8d956d0640 1 -- 172.21.15.19:0/3754110635 wait complete. 2024-09-19T04:06:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.211+0000 7f8d956d0640 1 Processor -- start 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.211+0000 7f8d956d0640 1 -- start start 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.211+0000 7f8d956d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d90101de0 0x7f8d90198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.211+0000 7f8d956d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90102790 0x7f8d901988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.211+0000 7f8d956d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d90106f10 0x7f8d9019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.211+0000 7f8d956d0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90111f70 con 0x7f8d90102790 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.211+0000 7f8d956d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90111df0 con 0x7f8d90101de0 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.211+0000 7f8d956d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901120f0 con 0x7f8d90106f10 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90102790 0x7f8d901988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90102790 0x7f8d901988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60172/0 (socket says 172.21.15.19:60172) 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 -- 172.21.15.19:0/1295613906 learned_addr learned my addr 172.21.15.19:0/1295613906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d90106f10 0x7f8d9019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8effd640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d90101de0 0x7f8d90198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 -- 172.21.15.19:0/1295613906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d90106f10 msgr2=0x7f8d9019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d90106f10 0x7f8d9019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 -- 172.21.15.19:0/1295613906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d90101de0 msgr2=0x7f8d90198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d90101de0 0x7f8d90198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 -- 172.21.15.19:0/1295613906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9019d340 con 0x7f8d90102790 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90102790 0x7f8d901988d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c920 tx=0x7f8d8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d77fff640 1 -- 172.21.15.19:0/1295613906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84007c70 con 0x7f8d90102790 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d956d0640 1 -- 172.21.15.19:0/1295613906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9019d630 con 0x7f8d90102790 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d77fff640 1 -- 172.21.15.19:0/1295613906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d84007e10 con 0x7f8d90102790 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d77fff640 1 -- 172.21.15.19:0/1295613906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84018890 con 0x7f8d90102790 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.212+0000 7f8d956d0640 1 -- 172.21.15.19:0/1295613906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d901a4f10 con 0x7f8d90102790 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d77fff640 1 -- 172.21.15.19:0/1295613906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8d84018a70 con 0x7f8d90102790 2024-09-19T04:06:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d97aa9740 1 Processor -- start 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d97aa9740 1 -- start start 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 0x560825be9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bea2f0 0x560825bf4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bf4bb0 0x560825bf6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560825b50090 con 0x560825bea2f0 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560825af7a70 con 0x560825be99d0 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560825b4c050 con 0x560825bf4bb0 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bea2f0 0x560825bf4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bea2f0 0x560825bf4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60184/0 (socket says 172.21.15.19:60184) 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d8effd640 1 -- 172.21.15.19:0/213978160 learned_addr learned my addr 172.21.15.19:0/213978160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.215+0000 7f8d77fff640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8d780a32d0 0x7f8d780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 0x560825be9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bf4bb0 0x560825bf6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8d780a32d0 0x7f8d780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8effd640 1 -- 172.21.15.19:0/213978160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bf4bb0 msgr2=0x560825bf6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8effd640 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bf4bb0 0x560825bf6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8effd640 1 -- 172.21.15.19:0/213978160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 msgr2=0x560825be9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8effd640 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 0x560825be9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8effd640 1 -- 172.21.15.19:0/213978160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560825b04130 con 0x560825bea2f0 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d77fff640 1 -- 172.21.15.19:0/1295613906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8d840cff30 con 0x7f8d90102790 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8effd640 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bea2f0 0x560825bf4670 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8d80033000 tx=0x7f8d80032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4035dd3e1fc9f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8d780a32d0 0x7f8d780a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005fd0 tx=0x7f8d88009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d757fa640 1 -- 172.21.15.19:0/213978160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8003f070 con 0x560825bea2f0 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d757fa640 1 -- 172.21.15.19:0/213978160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d800317b0 con 0x560825bea2f0 2024-09-19T04:06:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.216+0000 7f8d757fa640 1 -- 172.21.15.19:0/213978160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80035070 con 0x560825bea2f0 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 -- 172.21.15.19:0/213978160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bea2f0 msgr2=0x560825bf4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bea2f0 0x560825bf4670 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8d80033000 tx=0x7f8d80032ce0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 -- 172.21.15.19:0/213978160 shutdown_connections 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bf4bb0 0x560825bf6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bea2f0 0x560825bf4670 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/213978160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 0x560825be9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 -- 172.21.15.19:0/213978160 >> 172.21.15.19:0/213978160 conn(0x560825be5f60 msgr2=0x560825be6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 -- 172.21.15.19:0/213978160 shutdown_connections 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 -- 172.21.15.19:0/213978160 wait complete. 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 -- start start 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.217+0000 7f8d97aa9740 1 Processor -- start 2024-09-19T04:06:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825bf4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825bf1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bf1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560825bf7780 con 0x560825be99d0 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560825b0af20 con 0x560825bf4bb0 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560825bf7900 con 0x560825bea2f0 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d97aa9740 10 client.0 _ll_register_callbacks cb 0x560825ad4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825bf4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825bf4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60192/0 (socket says 172.21.15.19:60192) 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 -- 172.21.15.19:0/2952433531 learned_addr learned my addr 172.21.15.19:0/2952433531 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825bf1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bf1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 -- 172.21.15.19:0/2952433531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 msgr2=0x560825bf1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825bf1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 -- 172.21.15.19:0/2952433531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 msgr2=0x560825bf1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bf1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 -- 172.21.15.19:0/2952433531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608258c12b0 con 0x560825be99d0 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d8effd640 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825bf4410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005310 tx=0x7f8d80004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4300fdb6601ec3ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.218+0000 7f8d72ff5640 1 -- 172.21.15.19:0/2952433531 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8004a020 con 0x560825be99d0 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d72ff5640 1 -- 172.21.15.19:0/2952433531 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8d800035f0 con 0x560825be99d0 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 1 -- 172.21.15.19:0/2952433531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d800317b0 con 0x560825be99d0 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d72ff5640 1 -- 172.21.15.19:0/2952433531 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d80055050 con 0x560825be99d0 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 10 client.15105 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 1 -- 172.21.15.19:0/2952433531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560825c03070 con 0x560825be99d0 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d72ff5640 1 -- 172.21.15.19:0/2952433531 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d800317b0 con 0x560825be99d0 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d72ff5640 1 -- 172.21.15.19:0/2952433531 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8d80004bc0 con 0x560825be99d0 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 10 client.15105 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 20 client.15105 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560825c05230 0x560825c066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 4 client.15105 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.219+0000 7f8d97aa9740 1 -- 172.21.15.19:0/2952433531 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560825b5d600 con 0x560825c05230 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.220+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560825c05230 0x560825c066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.220+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560825c05230 0x560825c066a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1f66db3eb887ff server_cookie=3cd458313e839481 in_seq=0 out_seq=0 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.220+0000 7f8d72ff5640 10 client.15105 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:06:34.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d72ff5640 1 -- 172.21.15.19:0/2952433531 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560825b5d600 con 0x560825c05230 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d72ff5640 10 client.15105 handle_command_reply: tid=0 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 client.15105 shutdown 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 -- 172.21.15.19:0/2952433531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 msgr2=0x560825bf4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825bf4410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005310 tx=0x7f8d80004610 comp rx=0 tx=0).stop 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 -- 172.21.15.19:0/2952433531 shutdown_connections 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bf1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560825c05230 0x560825c066a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825bf1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/2952433531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825bf4410 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 -- 172.21.15.19:0/2952433531 >> 172.21.15.19:0/2952433531 conn(0x560825be5f60 msgr2=0x560825bebc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d72ff5640 0 client.15105 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:06:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 -- 172.21.15.19:0/2952433531 shutdown_connections 2024-09-19T04:06:34.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 1 -- 172.21.15.19:0/2952433531 wait complete. 2024-09-19T04:06:34.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.221+0000 7f8d97aa9740 20 client.15105 trim_cache size 0 max 16384 2024-09-19T04:06:34.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d97aa9740 1 Processor -- start 2024-09-19T04:06:34.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d97aa9740 1 -- start start 2024-09-19T04:06:34.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825c066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825beccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560825c049e0 con 0x560825be99d0 2024-09-19T04:06:34.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560825c050b0 con 0x560825bf4bb0 2024-09-19T04:06:34.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560825c05780 con 0x560825bea2f0 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825c066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825c066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60196/0 (socket says 172.21.15.19:60196) 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8effd640 1 -- 172.21.15.19:0/1921184832 learned_addr learned my addr 172.21.15.19:0/1921184832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825beccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8effd640 1 -- 172.21.15.19:0/1921184832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 msgr2=0x560825beccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8effd640 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825beccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8effd640 1 -- 172.21.15.19:0/1921184832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 msgr2=0x560825bee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8effd640 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8effd640 1 -- 172.21.15.19:0/1921184832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560825bea120 con 0x560825be99d0 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825beccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.239+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:34.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.240+0000 7f8d8effd640 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825c066a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8d80004fe0 tx=0x7f8d80005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d0b5cff35f58a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.240+0000 7f8d74ff9640 1 -- 172.21.15.19:0/1921184832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8004a040 con 0x560825be99d0 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.240+0000 7f8d74ff9640 1 -- 172.21.15.19:0/1921184832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80004ba0 con 0x560825be99d0 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.240+0000 7f8d74ff9640 1 -- 172.21.15.19:0/1921184832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8003f070 con 0x560825be99d0 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.240+0000 7f8d97aa9740 1 -- 172.21.15.19:0/1921184832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 msgr2=0x560825c066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.240+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825c066a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8d80004fe0 tx=0x7f8d80005720 comp rx=0 tx=0).stop 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 -- 172.21.15.19:0/1921184832 shutdown_connections 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825bf4bb0 0x560825bee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825beccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/1921184832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825be99d0 0x560825c066a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 -- 172.21.15.19:0/1921184832 >> 172.21.15.19:0/1921184832 conn(0x560825be5f60 msgr2=0x560825bec5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 -- 172.21.15.19:0/1921184832 shutdown_connections 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 -- 172.21.15.19:0/1921184832 wait complete. 2024-09-19T04:06:34.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 -- start start 2024-09-19T04:06:34.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 Processor -- start 2024-09-19T04:06:34.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 0x560825c0eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825c0f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bf4bb0 0x560825c07f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560825c06520 con 0x560825bf4bb0 2024-09-19T04:06:34.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560825c05e50 con 0x560825be99d0 2024-09-19T04:06:34.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560825c06bf0 con 0x560825bea2f0 2024-09-19T04:06:34.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d97aa9740 10 client.0 _ll_register_callbacks cb 0x7f8d80032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:34.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bf4bb0 0x560825c07f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bf4bb0 0x560825c07f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60202/0 (socket says 172.21.15.19:60202) 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.241+0000 7f8d8f7fe640 1 -- 172.21.15.19:0/3105130043 learned_addr learned my addr 172.21.15.19:0/3105130043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d8effd640 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 0x560825c0eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d8e7fc640 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825c0f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d8f7fe640 1 -- 172.21.15.19:0/3105130043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 msgr2=0x560825c0f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825c0f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d8f7fe640 1 -- 172.21.15.19:0/3105130043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 msgr2=0x560825c0eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 0x560825c0eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d8f7fe640 1 -- 172.21.15.19:0/3105130043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560825c08700 con 0x560825bf4bb0 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d8f7fe640 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bf4bb0 0x560825c07f20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d88010a10 tx=0x7f8d88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1bac0bbf0cac96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d72ff5640 1 -- 172.21.15.19:0/3105130043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88021c60 con 0x560825bf4bb0 2024-09-19T04:06:34.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d97aa9740 1 -- 172.21.15.19:0/3105130043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d80004ba0 con 0x560825bf4bb0 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d72ff5640 1 -- 172.21.15.19:0/3105130043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8d88021e00 con 0x560825bf4bb0 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d72ff5640 1 -- 172.21.15.19:0/3105130043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d88004b20 con 0x560825bf4bb0 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d97aa9740 10 client.15111 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.242+0000 7f8d97aa9740 1 -- 172.21.15.19:0/3105130043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560825c0d0a0 con 0x560825bf4bb0 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d72ff5640 1 -- 172.21.15.19:0/3105130043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80004ba0 con 0x560825bf4bb0 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d72ff5640 1 -- 172.21.15.19:0/3105130043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8d8800f040 con 0x560825bf4bb0 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d97aa9740 10 client.15111 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d97aa9740 10 client.15111 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d97aa9740 20 client.15111 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560825c73f90 0x560825c72ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d97aa9740 4 client.15111 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d97aa9740 1 -- 172.21.15.19:0/3105130043 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "status"}) -- 0x560825b57b10 con 0x560825c73f90 2024-09-19T04:06:34.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d8effd640 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560825c73f90 0x560825c72ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d8effd640 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560825c73f90 0x560825c72ca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439ed75a541e99f9 server_cookie=2550bcae19e43ddf in_seq=0 out_seq=0 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.243+0000 7f8d72ff5640 10 client.15111 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d72ff5640 1 -- 172.21.15.19:0/3105130043 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560825b57b10 con 0x560825c73f90 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d72ff5640 10 client.15111 handle_command_reply: tid=0 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 client.15111 shutdown 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 -- 172.21.15.19:0/3105130043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bf4bb0 msgr2=0x560825c07f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bf4bb0 0x560825c07f20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d88010a10 tx=0x7f8d88004a40 comp rx=0 tx=0).stop 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 -- 172.21.15.19:0/3105130043 shutdown_connections 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560825bf4bb0 0x560825c07f20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560825c73f90 0x560825c72ca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560825bea2f0 0x560825c0f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 --2- 172.21.15.19:0/3105130043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560825be99d0 0x560825c0eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 -- 172.21.15.19:0/3105130043 >> 172.21.15.19:0/3105130043 conn(0x560825be5f60 msgr2=0x560825bebc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d72ff5640 0 client.15111 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 -- 172.21.15.19:0/3105130043 shutdown_connections 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 1 -- 172.21.15.19:0/3105130043 wait complete. 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.244+0000 7f8d97aa9740 20 client.15111 trim_cache size 0 max 16384 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24476, 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:34.252 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:34.253 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:34.253 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 113.940658111, 2024-09-19T04:06:34.253 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:34.253 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 91, 2024-09-19T04:06:34.253 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:34.253 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 124.48375262099999 2024-09-19T04:06:34.253 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 -- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8d780a32d0 msgr2=0x7f8d780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8d780a32d0 0x7f8d780a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005fd0 tx=0x7f8d88009600 comp rx=0 tx=0).stop 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 -- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90102790 msgr2=0x7f8d901988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90102790 0x7f8d901988d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c920 tx=0x7f8d8400cde0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 -- 172.21.15.19:0/1295613906 shutdown_connections 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f8d780a32d0 0x7f8d780a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d90106f10 0x7f8d9019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d90102790 0x7f8d901988d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 --2- 172.21.15.19:0/1295613906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d90101de0 0x7f8d90198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 -- 172.21.15.19:0/1295613906 >> 172.21.15.19:0/1295613906 conn(0x7f8d900fd8c0 msgr2=0x7f8d900fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 -- 172.21.15.19:0/1295613906 shutdown_connections 2024-09-19T04:06:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:34.245+0000 7f8d956d0640 1 -- 172.21.15.19:0/1295613906 wait complete. 2024-09-19T04:06:34.939 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T04:06:34.939 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T04:06:34.950 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T04:06:34.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T04:06:35.206 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:35.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 -- 172.21.15.19:0/3725663647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c44104fd0 msgr2=0x7f0c441073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:35.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/3725663647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c44104fd0 0x7f0c441073c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009900 tx=0x7f0c3402a410 comp rx=0 tx=0).stop 2024-09-19T04:06:35.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 -- 172.21.15.19:0/3725663647 shutdown_connections 2024-09-19T04:06:35.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/3725663647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c44107900 0x7f0c44109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/3725663647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c44104fd0 0x7f0c441073c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/3725663647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c441026a0 0x7f0c44104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 -- 172.21.15.19:0/3725663647 >> 172.21.15.19:0/3725663647 conn(0x7f0c440fc2a0 msgr2=0x7f0c440fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:35.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 -- 172.21.15.19:0/3725663647 shutdown_connections 2024-09-19T04:06:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.660+0000 7f0c4ac99640 1 -- 172.21.15.19:0/3725663647 wait complete. 2024-09-19T04:06:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c4ac99640 1 Processor -- start 2024-09-19T04:06:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c4ac99640 1 -- start start 2024-09-19T04:06:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c4ac99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c441026a0 0x7f0c44102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c4ac99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c44104fd0 0x7f0c44100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c4ac99640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c44107900 0x7f0c44100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c4ac99640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4410d8f0 con 0x7f0c441026a0 2024-09-19T04:06:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c4ac99640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4410d770 con 0x7f0c44107900 2024-09-19T04:06:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c4ac99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4410da70 con 0x7f0c44104fd0 2024-09-19T04:06:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c48a35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c441026a0 0x7f0c44102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c48a35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c441026a0 0x7f0c44102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60232/0 (socket says 172.21.15.19:60232) 2024-09-19T04:06:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.661+0000 7f0c48a35640 1 -- 172.21.15.19:0/2298020106 learned_addr learned my addr 172.21.15.19:0/2298020106 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c3bfff640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c44104fd0 0x7f0c44100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c49236640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c44107900 0x7f0c44100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c48a35640 1 -- 172.21.15.19:0/2298020106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c44104fd0 msgr2=0x7f0c44100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:35.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c48a35640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c44104fd0 0x7f0c44100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c48a35640 1 -- 172.21.15.19:0/2298020106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c44107900 msgr2=0x7f0c44100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c48a35640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c44107900 0x7f0c44100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c48a35640 1 -- 172.21.15.19:0/2298020106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c44101450 con 0x7f0c441026a0 2024-09-19T04:06:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c49236640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c44107900 0x7f0c44100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c48a35640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c441026a0 0x7f0c44102000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c890 tx=0x7f0c4000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c39ffb640 1 -- 172.21.15.19:0/2298020106 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40004220 con 0x7f0c441026a0 2024-09-19T04:06:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c39ffb640 1 -- 172.21.15.19:0/2298020106 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c400043c0 con 0x7f0c441026a0 2024-09-19T04:06:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c441a2130 con 0x7f0c441026a0 2024-09-19T04:06:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c39ffb640 1 -- 172.21.15.19:0/2298020106 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c400049e0 con 0x7f0c441026a0 2024-09-19T04:06:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.662+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c441a2710 con 0x7f0c441026a0 2024-09-19T04:06:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.663+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c440fdd90 con 0x7f0c441026a0 2024-09-19T04:06:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.664+0000 7f0c39ffb640 1 -- 172.21.15.19:0/2298020106 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0c400079b0 con 0x7f0c441026a0 2024-09-19T04:06:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.665+0000 7f0c39ffb640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0c2c0a3350 0x7f0c2c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.665+0000 7f0c39ffb640 1 -- 172.21.15.19:0/2298020106 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f0c400cfed0 con 0x7f0c441026a0 2024-09-19T04:06:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.665+0000 7f0c3bfff640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0c2c0a3350 0x7f0c2c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.666+0000 7f0c3bfff640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0c2c0a3350 0x7f0c2c0a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c34006010 tx=0x7f0c34000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.669+0000 7f0c39ffb640 1 -- 172.21.15.19:0/2298020106 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c40014030 con 0x7f0c441026a0 2024-09-19T04:06:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.861+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0c44062630 con 0x7f0c2c0a3350 2024-09-19T04:06:35.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:35 smithi019 ceph-mon[30938]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.863+0000 7f0c39ffb640 1 -- 172.21.15.19:0/2298020106 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0c44062630 con 0x7f0c2c0a3350 2024-09-19T04:06:35.864 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-09-19T04:06:35.864 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-09-19T04:06:35.864 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-09-19T04:06:35.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0c2c0a3350 msgr2=0x7f0c2c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:35.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0c2c0a3350 0x7f0c2c0a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c34006010 tx=0x7f0c34000bc0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c441026a0 msgr2=0x7f0c44102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:35.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c441026a0 0x7f0c44102000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c890 tx=0x7f0c4000cd50 comp rx=0 tx=0).stop 2024-09-19T04:06:35.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 shutdown_connections 2024-09-19T04:06:35.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0c2c0a3350 0x7f0c2c0a5810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c44107900 0x7f0c44100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c44104fd0 0x7f0c44100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 --2- 172.21.15.19:0/2298020106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c441026a0 0x7f0c44102000 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:35.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 >> 172.21.15.19:0/2298020106 conn(0x7f0c440fc2a0 msgr2=0x7f0c44102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:35.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 shutdown_connections 2024-09-19T04:06:35.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:35.865+0000 7f0c4ac99640 1 -- 172.21.15.19:0/2298020106 wait complete. 2024-09-19T04:06:35.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:35 smithi111 ceph-mon[35365]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:35.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:35 smithi196 ceph-mon[35731]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:36.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T04:06:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:36.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:36 smithi111 ceph-mon[35365]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:36.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:36 smithi019 ceph-mon[30938]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:36.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:36 smithi196 ceph-mon[35731]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 -- 172.21.15.19:0/3165068995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0102690 msgr2=0x7f77c0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 --2- 172.21.15.19:0/3165068995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0102690 0x7f77c0108f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009ca0 tx=0x7f77b802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 -- 172.21.15.19:0/3165068995 shutdown_connections 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 --2- 172.21.15.19:0/3165068995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0102690 0x7f77c0108f20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 --2- 172.21.15.19:0/3165068995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c0101cd0 0x7f77c0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 --2- 172.21.15.19:0/3165068995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c0100ad0 0x7f77c0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 -- 172.21.15.19:0/3165068995 >> 172.21.15.19:0/3165068995 conn(0x7f77c00fc280 msgr2=0x7f77c00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 -- 172.21.15.19:0/3165068995 shutdown_connections 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.394+0000 7f77c5db7640 1 -- 172.21.15.19:0/3165068995 wait complete. 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.395+0000 7f77c5db7640 1 Processor -- start 2024-09-19T04:06:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.395+0000 7f77c5db7640 1 -- start start 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.395+0000 7f77c5db7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c0100ad0 0x7f77c0104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.395+0000 7f77c5db7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0101cd0 0x7f77c01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.395+0000 7f77c5db7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c0102690 0x7f77c01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77c5db7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010c730 con 0x7f77c0101cd0 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77c5db7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010c5b0 con 0x7f77c0100ad0 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77c5db7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010c8b0 con 0x7f77c0102690 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0101cd0 0x7f77c01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0101cd0 0x7f77c01030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60242/0 (socket says 172.21.15.19:60242) 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 -- 172.21.15.19:0/339324970 learned_addr learned my addr 172.21.15.19:0/339324970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77bf7fe640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c0100ad0 0x7f77c0104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77bffff640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c0102690 0x7f77c01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 -- 172.21.15.19:0/339324970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c0102690 msgr2=0x7f77c01035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c0102690 0x7f77c01035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 -- 172.21.15.19:0/339324970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c0100ad0 msgr2=0x7f77c0104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c0100ad0 0x7f77c0104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 -- 172.21.15.19:0/339324970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c0103eb0 con 0x7f77c0101cd0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77bf7fe640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c0100ad0 0x7f77c0104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77b6ffd640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0101cd0 0x7f77c01030b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c920 tx=0x7f77a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77bd7fa640 1 -- 172.21.15.19:0/339324970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8007c70 con 0x7f77c0101cd0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77bd7fa640 1 -- 172.21.15.19:0/339324970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a8007e10 con 0x7f77c0101cd0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77bd7fa640 1 -- 172.21.15.19:0/339324970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8018890 con 0x7f77c0101cd0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c01a1fa0 con 0x7f77c0101cd0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.396+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c01a2400 con 0x7f77c0101cd0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.397+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c01a2690 con 0x7f77c0101cd0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.398+0000 7f77bd7fa640 1 -- 172.21.15.19:0/339324970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f77a8018a70 con 0x7f77c0101cd0 2024-09-19T04:06:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.399+0000 7f77bd7fa640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f77ac0a3350 0x7f77ac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.399+0000 7f77bf7fe640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f77ac0a3350 0x7f77ac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.399+0000 7f77bd7fa640 1 -- 172.21.15.19:0/339324970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f77a80cff40 con 0x7f77c0101cd0 2024-09-19T04:06:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.400+0000 7f77bf7fe640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f77ac0a3350 0x7f77ac0a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77c0101b30 tx=0x7f77b000a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.404+0000 7f77bd7fa640 1 -- 172.21.15.19:0/339324970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f77a8014030 con 0x7f77c0101cd0 2024-09-19T04:06:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.588+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f77c01a2860 con 0x7f77ac0a3350 2024-09-19T04:06:37.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.601+0000 7f77bd7fa640 1 -- 172.21.15.19:0/339324970 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f77c01a2860 con 0x7f77ac0a3350 2024-09-19T04:06:37.603 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T04:06:37.603 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (2m) 60s ago 2m 15.5M - 19.1.1-361-g2f599a46 d2baa855c067 d719fb1c5e70 2024-09-19T04:06:37.603 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi111 running (2m) 61s ago 2m 26.4M - 19.1.1-361-g2f599a46 d2baa855c067 f4d3c6ec631b 2024-09-19T04:06:37.603 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi196 running (2m) 61s ago 2m 17.2M - 19.1.1-361-g2f599a46 d2baa855c067 12a2b6b5c202 2024-09-19T04:06:37.603 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (2m) 60s ago 2m 13.3M - 19.1.1-361-g2f599a46 d2baa855c067 a4342fddd53d 2024-09-19T04:06:37.603 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi111 running (2m) 61s ago 2m 16.4M - 19.1.1-361-g2f599a46 d2baa855c067 afee5bda59ab 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi196 running (2m) 61s ago 2m 17.3M - 19.1.1-361-g2f599a46 d2baa855c067 e31700601e2a 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (2m) 60s ago 2m 15.4M - 19.1.1-361-g2f599a46 d2baa855c067 2d75d5ec4ee0 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi111 running (2m) 61s ago 2m 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 affc28ae2aa2 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi196 running (2m) 61s ago 2m 18.7M - 19.1.1-361-g2f599a46 d2baa855c067 c1594205ad50 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (2m) 60s ago 2m 14.5M - 19.1.1-361-g2f599a46 d2baa855c067 6387b03525af 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi111 running (2m) 61s ago 2m 15.6M - 19.1.1-361-g2f599a46 d2baa855c067 120cf7baa660 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi196 running (2m) 61s ago 2m 17.9M - 19.1.1-361-g2f599a46 d2baa855c067 ec465ec7efa5 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (9m) 60s ago 9m 507M - 19.1.1-361-g2f599a46 d2baa855c067 ffebeb9c157e 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi111 *:8443,8765 running (7m) 61s ago 7m 444M - 19.1.1-361-g2f599a46 d2baa855c067 539ccb18f7b0 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi196 *:8443,8765 running (8m) 61s ago 8m 442M - 19.1.1-361-g2f599a46 d2baa855c067 1dfea3d07311 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (9m) 60s ago 9m 49.8M 2048M 19.1.1-361-g2f599a46 d2baa855c067 4593a6e9f16a 2024-09-19T04:06:37.604 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi111 running (8m) 61s ago 8m 39.6M 2048M 19.1.1-361-g2f599a46 d2baa855c067 5e9a143ee94e 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi196 running (8m) 61s ago 8m 40.1M 2048M 19.1.1-361-g2f599a46 d2baa855c067 9a1e0c7faa3b 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (7m) 60s ago 7m 46.3M 4273M 19.1.1-361-g2f599a46 d2baa855c067 93ade25bb40e 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi111 running (7m) 61s ago 7m 49.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b8c7f6dce874 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi196 running (6m) 61s ago 6m 47.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 9ffcb7f3c614 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (6m) 60s ago 6m 46.6M 4273M 19.1.1-361-g2f599a46 d2baa855c067 41894d07bdfe 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi111 running (5m) 61s ago 5m 42.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 19198329ee78 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi196 running (5m) 61s ago 5m 58.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 f346cd81e199 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (5m) 60s ago 5m 43.5M 4273M 19.1.1-361-g2f599a46 d2baa855c067 54b3b724bf61 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi111 running (4m) 61s ago 4m 56.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 91ae58c61dd6 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi196 running (4m) 61s ago 4m 54.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 0652d2806106 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (3m) 60s ago 3m 45.3M 4273M 19.1.1-361-g2f599a46 d2baa855c067 a152060d7b29 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi111 running (3m) 61s ago 3m 45.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 589f208e62ea 2024-09-19T04:06:37.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi196 running (2m) 61s ago 2m 42.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 a688b07d0cfb 2024-09-19T04:06:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f77ac0a3350 msgr2=0x7f77ac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f77ac0a3350 0x7f77ac0a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77c0101b30 tx=0x7f77b000a560 comp rx=0 tx=0).stop 2024-09-19T04:06:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0101cd0 msgr2=0x7f77c01030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0101cd0 0x7f77c01030b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c920 tx=0x7f77a800cde0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 shutdown_connections 2024-09-19T04:06:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f77ac0a3350 0x7f77ac0a5810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c0102690 0x7f77c01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77c0101cd0 0x7f77c01030b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 --2- 172.21.15.19:0/339324970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77c0100ad0 0x7f77c0104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 >> 172.21.15.19:0/339324970 conn(0x7f77c00fc280 msgr2=0x7f77c00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 shutdown_connections 2024-09-19T04:06:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:37.603+0000 7f77c5db7640 1 -- 172.21.15.19:0/339324970 wait complete. 2024-09-19T04:06:37.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:37 smithi019 ceph-mon[30938]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:37.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:37 smithi111 ceph-mon[35365]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:37.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:37 smithi196 ceph-mon[35731]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:06:38.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T04:06:38.663 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:38.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:38 smithi111 ceph-mon[35365]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:38.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:38 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:38 smithi196 ceph-mon[35731]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:38 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:38.912 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:38 smithi019 ceph-mon[30938]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:38.912 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:38 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:06:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 -- 172.21.15.19:0/4221781547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c105010 msgr2=0x7fba1c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 --2- 172.21.15.19:0/4221781547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c105010 0x7fba1c107400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba10009900 tx=0x7fba1002a410 comp rx=0 tx=0).stop 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 -- 172.21.15.19:0/4221781547 shutdown_connections 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 --2- 172.21.15.19:0/4221781547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba1c107940 0x7fba1c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 --2- 172.21.15.19:0/4221781547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c105010 0x7fba1c107400 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 --2- 172.21.15.19:0/4221781547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1c1026e0 0x7fba1c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 -- 172.21.15.19:0/4221781547 >> 172.21.15.19:0/4221781547 conn(0x7fba1c0fc2a0 msgr2=0x7fba1c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 -- 172.21.15.19:0/4221781547 shutdown_connections 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 -- 172.21.15.19:0/4221781547 wait complete. 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.767+0000 7fba213ef640 1 Processor -- start 2024-09-19T04:06:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba213ef640 1 -- start start 2024-09-19T04:06:39.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba213ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba1c1026e0 0x7fba1c102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:39.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba213ef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c105010 0x7fba1c100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba213ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1c107940 0x7fba1c100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba213ef640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c10d8e0 con 0x7fba1c105010 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba213ef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c10d760 con 0x7fba1c1026e0 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba213ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c10da60 con 0x7fba1c107940 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba1c1026e0 0x7fba1c102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c105010 0x7fba1c100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba1c1026e0 0x7fba1c102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:42958/0 (socket says 172.21.15.19:42958) 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1affd640 1 -- 172.21.15.19:0/3846497226 learned_addr learned my addr 172.21.15.19:0/3846497226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1b7fe640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1c107940 0x7fba1c100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1affd640 1 -- 172.21.15.19:0/3846497226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1c107940 msgr2=0x7fba1c100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1affd640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1c107940 0x7fba1c100bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1affd640 1 -- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c105010 msgr2=0x7fba1c100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1affd640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c105010 0x7fba1c100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.768+0000 7fba1affd640 1 -- 172.21.15.19:0/3846497226 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c101490 con 0x7fba1c1026e0 2024-09-19T04:06:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.769+0000 7fba1affd640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba1c1026e0 0x7fba1c102040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00d770 tx=0x7fba0c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.770+0000 7fba03fff640 1 -- 172.21.15.19:0/3846497226 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c017070 con 0x7fba1c1026e0 2024-09-19T04:06:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.770+0000 7fba03fff640 1 -- 172.21.15.19:0/3846497226 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0c005500 con 0x7fba1c1026e0 2024-09-19T04:06:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.770+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c1a20d0 con 0x7fba1c1026e0 2024-09-19T04:06:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.770+0000 7fba03fff640 1 -- 172.21.15.19:0/3846497226 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c004e00 con 0x7fba1c1026e0 2024-09-19T04:06:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.771+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c1a25e0 con 0x7fba1c1026e0 2024-09-19T04:06:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.772+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c0fdd90 con 0x7fba1c1026e0 2024-09-19T04:06:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.773+0000 7fba03fff640 1 -- 172.21.15.19:0/3846497226 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fba0c010460 con 0x7fba1c1026e0 2024-09-19T04:06:39.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.774+0000 7fba03fff640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fba040a3350 0x7fba040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:39.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.774+0000 7fba1a7fc640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fba040a3350 0x7fba040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:39.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.774+0000 7fba03fff640 1 -- 172.21.15.19:0/3846497226 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fba0c0cf7a0 con 0x7fba1c1026e0 2024-09-19T04:06:39.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.774+0000 7fba1a7fc640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fba040a3350 0x7fba040a5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba1002aa00 tx=0x7fba100315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:39.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.778+0000 7fba03fff640 1 -- 172.21.15.19:0/3846497226 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fba0c09b970 con 0x7fba1c1026e0 2024-09-19T04:06:39.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:39 smithi111 ceph-mon[35365]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:39.931 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:39 smithi019 ceph-mon[30938]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:39.958 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:39 smithi196 ceph-mon[35731]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:06:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.980+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fba1c062630 con 0x7fba040a3350 2024-09-19T04:06:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.986+0000 7fba03fff640 1 -- 172.21.15.19:0/3846497226 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fba1c062630 con 0x7fba040a3350 2024-09-19T04:06:39.989 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T04:06:39.989 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 62s ago 2m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-19T04:06:39.989 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 62s ago 8m smithi019=x;smithi111=y;smithi196=z;count:3 2024-09-19T04:06:39.989 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 62s ago 8m smithi019:172.21.15.19=a;smithi111:172.21.15.111=b;smithi196:172.21.15.196=c;count:3 2024-09-19T04:06:39.989 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 62s ago - 2024-09-19T04:06:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fba040a3350 msgr2=0x7fba040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fba040a3350 0x7fba040a5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba1002aa00 tx=0x7fba100315b0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba1c1026e0 msgr2=0x7fba1c102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba1c1026e0 0x7fba1c102040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00d770 tx=0x7fba0c00dc30 comp rx=0 tx=0).stop 2024-09-19T04:06:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 shutdown_connections 2024-09-19T04:06:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fba040a3350 0x7fba040a5810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1c107940 0x7fba1c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c105010 0x7fba1c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 --2- 172.21.15.19:0/3846497226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba1c1026e0 0x7fba1c102040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 >> 172.21.15.19:0/3846497226 conn(0x7fba1c0fc2a0 msgr2=0x7fba1c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 shutdown_connections 2024-09-19T04:06:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:39.989+0000 7fba213ef640 1 -- 172.21.15.19:0/3846497226 wait complete. 2024-09-19T04:06:40.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T04:06:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:40 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:40 smithi111 ceph-mon[35365]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:40 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:40 smithi019 ceph-mon[30938]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:40 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:41.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:40 smithi196 ceph-mon[35731]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:41.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3883071342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0102e00 msgr2=0x7f36b0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3883071342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0102e00 0x7f36b0103280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009900 tx=0x7f36a002a410 comp rx=0 tx=0).stop 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3883071342 shutdown_connections 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3883071342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36b01037c0 0x7f36b010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3883071342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0102e00 0x7f36b0103280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3883071342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36b0101c00 0x7f36b0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3883071342 >> 172.21.15.19:0/3883071342 conn(0x7f36b00fd390 msgr2=0x7f36b00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3883071342 shutdown_connections 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.568+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3883071342 wait complete. 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b7f9c640 1 Processor -- start 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b7f9c640 1 -- start start 2024-09-19T04:06:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b7f9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0101c00 0x7f36b0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b7f9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36b0102e00 0x7f36b01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b7f9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36b01037c0 0x7f36b019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b7f9c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36b010d8b0 con 0x7f36b0101c00 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b7f9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f36b010d730 con 0x7f36b0102e00 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b7f9c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36b010da30 con 0x7f36b01037c0 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b5d38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0101c00 0x7f36b0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b5d38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0101c00 0x7f36b0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57748/0 (socket says 172.21.15.19:57748) 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b5d38640 1 -- 172.21.15.19:0/3350495364 learned_addr learned my addr 172.21.15.19:0/3350495364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b6539640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36b01037c0 0x7f36b019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b5537640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36b0102e00 0x7f36b01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b5d38640 1 -- 172.21.15.19:0/3350495364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36b01037c0 msgr2=0x7f36b019f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b5d38640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36b01037c0 0x7f36b019f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b5d38640 1 -- 172.21.15.19:0/3350495364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36b0102e00 msgr2=0x7f36b01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.569+0000 7f36b5d38640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36b0102e00 0x7f36b01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.570+0000 7f36b5d38640 1 -- 172.21.15.19:0/3350495364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b019fae0 con 0x7f36b0101c00 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.570+0000 7f36b5d38640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0101c00 0x7f36b0197eb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00d770 tx=0x7f36ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.570+0000 7f36a6ffd640 1 -- 172.21.15.19:0/3350495364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac017070 con 0x7f36b0101c00 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.570+0000 7f36a6ffd640 1 -- 172.21.15.19:0/3350495364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ac009c30 con 0x7f36b0101c00 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.570+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b019fdd0 con 0x7f36b0101c00 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.570+0000 7f36a6ffd640 1 -- 172.21.15.19:0/3350495364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac003dd0 con 0x7f36b0101c00 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.570+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b01a0230 con 0x7f36b0101c00 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.571+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b0108fb0 con 0x7f36b0101c00 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.572+0000 7f36a6ffd640 1 -- 172.21.15.19:0/3350495364 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f36ac003f70 con 0x7f36b0101c00 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.573+0000 7f36a6ffd640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f369c0a32f0 0x7f369c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.573+0000 7f36b5537640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f369c0a32f0 0x7f369c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.573+0000 7f36a6ffd640 1 -- 172.21.15.19:0/3350495364 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f36ac0cf4d0 con 0x7f36b0101c00 2024-09-19T04:06:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.573+0000 7f36b5537640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f369c0a32f0 0x7f369c0a57b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36a0031440 tx=0x7f36a00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.577+0000 7f36a6ffd640 1 -- 172.21.15.19:0/3350495364 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f36ac009de0 con 0x7f36b0101c00 2024-09-19T04:06:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.759+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f36b0062630 con 0x7f369c0a32f0 2024-09-19T04:06:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.760+0000 7f36a6ffd640 1 -- 172.21.15.19:0/3350495364 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f36b0062630 con 0x7f369c0a32f0 2024-09-19T04:06:41.762 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-09-19T04:06:41.762 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-09-19T04:06:41.763 INFO:teuthology.orchestra.run.smithi019.stdout:smithi111 172.21.15.111 2024-09-19T04:06:41.763 INFO:teuthology.orchestra.run.smithi019.stdout:smithi196 172.21.15.196 2024-09-19T04:06:41.763 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f369c0a32f0 msgr2=0x7f369c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f369c0a32f0 0x7f369c0a57b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36a0031440 tx=0x7f36a00315b0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0101c00 msgr2=0x7f36b0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0101c00 0x7f36b0197eb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00d770 tx=0x7f36ac00dc30 comp rx=0 tx=0).stop 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 shutdown_connections 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f369c0a32f0 0x7f369c0a57b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36b01037c0 0x7f36b019f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36b0102e00 0x7f36b01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 --2- 172.21.15.19:0/3350495364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b0101c00 0x7f36b0197eb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.762+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 >> 172.21.15.19:0/3350495364 conn(0x7f36b00fd390 msgr2=0x7f36b0106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.763+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 shutdown_connections 2024-09-19T04:06:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:41.763+0000 7f36b7f9c640 1 -- 172.21.15.19:0/3350495364 wait complete. 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]: dispatch 2024-09-19T04:06:42.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:06:42.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:06:42.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:41 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:06:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-19T04:06:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-09-19T04:06:42.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-09-19T04:06:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]: dispatch 2024-09-19T04:06:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:06:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:06:42.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:41 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:06:42.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]: dispatch 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:06:42.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:41 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:06:42.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T04:06:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:43.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:42 smithi019 ceph-mon[30938]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:43.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-19T04:06:43.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-09-19T04:06:43.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-19T04:06:43.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]': finished 2024-09-19T04:06:43.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:42 smithi019 ceph-mon[30938]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:06:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:42 smithi196 ceph-mon[35731]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-19T04:06:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-09-19T04:06:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-19T04:06:43.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]': finished 2024-09-19T04:06:43.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:42 smithi196 ceph-mon[35731]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:06:43.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.191+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2698321385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100dd0 msgr2=0x7f9e98101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:43.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.191+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2698321385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100dd0 0x7f9e98101230 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009900 tx=0x7f9e8802a410 comp rx=0 tx=0).stop 2024-09-19T04:06:43.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.191+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2698321385 shutdown_connections 2024-09-19T04:06:43.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.191+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2698321385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e98101770 0x7f9e98101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.191+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2698321385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100dd0 0x7f9e98101230 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.191+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2698321385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e98100490 0x7f9e98100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.191+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2698321385 >> 172.21.15.19:0/2698321385 conn(0x7f9e980fc2a0 msgr2=0x7f9e980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:43.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2698321385 shutdown_connections 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2698321385 wait complete. 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 Processor -- start 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 -- start start 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100490 0x7f9e9819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e98100dd0 0x7f9e9819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e98101770 0x7f9e981a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9810fb50 con 0x7f9e98100490 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9810f9d0 con 0x7f9e98101770 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9f59c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9810fcd0 con 0x7f9e98100dd0 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.192+0000 7f9e9d338640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100490 0x7f9e9819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9d338640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100490 0x7f9e9819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57764/0 (socket says 172.21.15.19:57764) 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9d338640 1 -- 172.21.15.19:0/2500945435 learned_addr learned my addr 172.21.15.19:0/2500945435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9cb37640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e98100dd0 0x7f9e9819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:43.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9db39640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e98101770 0x7f9e981a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9d338640 1 -- 172.21.15.19:0/2500945435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e98100dd0 msgr2=0x7f9e9819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9d338640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e98100dd0 0x7f9e9819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9d338640 1 -- 172.21.15.19:0/2500945435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e98101770 msgr2=0x7f9e981a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9d338640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e98101770 0x7f9e981a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9d338640 1 -- 172.21.15.19:0/2500945435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e981a1de0 con 0x7f9e98100490 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9d338640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100490 0x7f9e9819a1b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c970 tx=0x7f9e9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e8e7fc640 1 -- 172.21.15.19:0/2500945435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94007bb0 con 0x7f9e98100490 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e8e7fc640 1 -- 172.21.15.19:0/2500945435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e94007d50 con 0x7f9e98100490 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e981a20d0 con 0x7f9e98100490 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e8e7fc640 1 -- 172.21.15.19:0/2500945435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400f450 con 0x7f9e98100490 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.193+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e981a2610 con 0x7f9e98100490 2024-09-19T04:06:43.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.194+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e9810b1e0 con 0x7f9e98100490 2024-09-19T04:06:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.195+0000 7f9e8e7fc640 1 -- 172.21.15.19:0/2500945435 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9e94004fe0 con 0x7f9e98100490 2024-09-19T04:06:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.196+0000 7f9e8e7fc640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9e840a3270 0x7f9e840a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.196+0000 7f9e8e7fc640 1 -- 172.21.15.19:0/2500945435 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f9e940cfb80 con 0x7f9e98100490 2024-09-19T04:06:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.196+0000 7f9e9cb37640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9e840a3270 0x7f9e840a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.197+0000 7f9e9cb37640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9e840a3270 0x7f9e840a5730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9e88006010 tx=0x7f9e88031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.200+0000 7f9e8e7fc640 1 -- 172.21.15.19:0/2500945435 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9e94014030 con 0x7f9e98100490 2024-09-19T04:06:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.384+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9e98062630 con 0x7f9e840a3270 2024-09-19T04:06:43.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.386+0000 7f9e8e7fc640 1 -- 172.21.15.19:0/2500945435 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9e98062630 con 0x7f9e840a3270 2024-09-19T04:06:43.388 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T04:06:43.389 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T04:06:43.389 INFO:teuthology.orchestra.run.smithi019.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T04:06:43.389 INFO:teuthology.orchestra.run.smithi019.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T04:06:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.388+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9e840a3270 msgr2=0x7f9e840a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.388+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9e840a3270 0x7f9e840a5730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9e88006010 tx=0x7f9e88031400 comp rx=0 tx=0).stop 2024-09-19T04:06:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.388+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100490 msgr2=0x7f9e9819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.388+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100490 0x7f9e9819a1b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c970 tx=0x7f9e9400ce30 comp rx=0 tx=0).stop 2024-09-19T04:06:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.389+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 shutdown_connections 2024-09-19T04:06:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.389+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9e840a3270 0x7f9e840a5730 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.389+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e98101770 0x7f9e981a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.389+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e98100dd0 0x7f9e9819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.389+0000 7f9e9f59c640 1 --2- 172.21.15.19:0/2500945435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e98100490 0x7f9e9819a1b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:43.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.389+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 >> 172.21.15.19:0/2500945435 conn(0x7f9e980fc2a0 msgr2=0x7f9e980fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:43.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.389+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 shutdown_connections 2024-09-19T04:06:43.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:43.389+0000 7f9e9f59c640 1 -- 172.21.15.19:0/2500945435 wait complete. 2024-09-19T04:06:43.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:42 smithi111 ceph-mon[35365]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:43.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-19T04:06:43.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-09-19T04:06:43.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-19T04:06:43.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]': finished 2024-09-19T04:06:43.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:42 smithi111 ceph-mon[35365]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T04:06:44.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:43 smithi019 ceph-mon[30938]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:06:44.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:43 smithi019 ceph-mon[30938]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:06:44.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:43 smithi019 ceph-mon[30938]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:44.133 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T04:06:44.142 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T04:06:44.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T04:06:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:43 smithi196 ceph-mon[35731]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:06:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:43 smithi196 ceph-mon[35731]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:06:44.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:43 smithi196 ceph-mon[35731]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:43 smithi111 ceph-mon[35365]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T04:06:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:43 smithi111 ceph-mon[35365]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T04:06:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:43 smithi111 ceph-mon[35365]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:06:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:44.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 -- 172.21.15.19:0/3358979096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 msgr2=0x7fa010108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 --2- 172.21.15.19:0/3358979096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 0x7fa010108f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa008009ca0 tx=0x7fa00802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 -- 172.21.15.19:0/3358979096 shutdown_connections 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 --2- 172.21.15.19:0/3358979096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 0x7fa010108f20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 --2- 172.21.15.19:0/3358979096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa010101cd0 0x7fa010102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 --2- 172.21.15.19:0/3358979096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa010100ad0 0x7fa010100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 -- 172.21.15.19:0/3358979096 >> 172.21.15.19:0/3358979096 conn(0x7fa0100fc280 msgr2=0x7fa0100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 -- 172.21.15.19:0/3358979096 shutdown_connections 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.839+0000 7fa017f60640 1 -- 172.21.15.19:0/3358979096 wait complete. 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa017f60640 1 Processor -- start 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa017f60640 1 -- start start 2024-09-19T04:06:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa017f60640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa010100ad0 0x7fa010197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa017f60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa010101cd0 0x7fa0101983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa017f60640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 0x7fa01019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa017f60640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa01010d880 con 0x7fa010102690 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa017f60640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa01010d700 con 0x7fa010100ad0 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa017f60640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa01010da00 con 0x7fa010101cd0 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0164fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 0x7fa01019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0164fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 0x7fa01019f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57776/0 (socket says 172.21.15.19:57776) 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0164fd640 1 -- 172.21.15.19:0/2173677755 learned_addr learned my addr 172.21.15.19:0/2173677755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa015cfc640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa010100ad0 0x7fa010197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0154fb640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa010101cd0 0x7fa0101983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0164fd640 1 -- 172.21.15.19:0/2173677755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa010101cd0 msgr2=0x7fa0101983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0164fd640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa010101cd0 0x7fa0101983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0164fd640 1 -- 172.21.15.19:0/2173677755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa010100ad0 msgr2=0x7fa010197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0164fd640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa010100ad0 0x7fa010197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0164fd640 1 -- 172.21.15.19:0/2173677755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01019fae0 con 0x7fa010102690 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa015cfc640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa010100ad0 0x7fa010197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.840+0000 7fa0154fb640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa010101cd0 0x7fa0101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.841+0000 7fa0164fd640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 0x7fa01019f470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa008005e30 tx=0x7fa008031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.841+0000 7fa006ffd640 1 -- 172.21.15.19:0/2173677755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008003c30 con 0x7fa010102690 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.841+0000 7fa006ffd640 1 -- 172.21.15.19:0/2173677755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa008003dd0 con 0x7fa010102690 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.841+0000 7fa006ffd640 1 -- 172.21.15.19:0/2173677755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00802fde0 con 0x7fa010102690 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.841+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01019fd70 con 0x7fa010102690 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.841+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0101a0250 con 0x7fa010102690 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.842+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa010107e70 con 0x7fa010102690 2024-09-19T04:06:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.843+0000 7fa006ffd640 1 -- 172.21.15.19:0/2173677755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa008004050 con 0x7fa010102690 2024-09-19T04:06:44.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.844+0000 7fa006ffd640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9ffc0a3270 0x7f9ffc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:44.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.844+0000 7fa015cfc640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9ffc0a3270 0x7f9ffc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:44.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.844+0000 7fa006ffd640 1 -- 172.21.15.19:0/2173677755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fa0080f1130 con 0x7fa010102690 2024-09-19T04:06:44.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.844+0000 7fa015cfc640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9ffc0a3270 0x7f9ffc0a5730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa010101b30 tx=0x7fa00c00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:44.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:44.848+0000 7fa006ffd640 1 -- 172.21.15.19:0/2173677755 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0080bdab0 con 0x7fa010102690 2024-09-19T04:06:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.064+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa010062e90 con 0x7fa010102690 2024-09-19T04:06:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.064+0000 7fa006ffd640 1 -- 172.21.15.19:0/2173677755 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3134 (secure 0 0 0) 0x7fa0080bd3b0 con 0x7fa010102690 2024-09-19T04:06:45.067 INFO:teuthology.orchestra.run.smithi019.stdout:e28 2024-09-19T04:06:45.067 INFO:teuthology.orchestra.run.smithi019.stdout:btime 2024-09-19T04:05:23:602377+0000 2024-09-19T04:06:45.067 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:06:45.067 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:06:45.067 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-09-19T04:06:45.067 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:45.067 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-09-19T04:06:45.067 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 28 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-09-19T04:04:40.077653+0000 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-09-19T04:05:23.279052+0000 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-09-19T04:06:45.068 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 5 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1,2,3,4 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24476,1=24469,2=24454,3=24452,4=24448} 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3,4] 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-09-19T04:06:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24476 members: 24476,24469,24454,24452,24448 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{0:24476} state up:active seq 4 addr [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{0:24437} state up:standby-replay seq 1 addr [v2:172.21.15.111:6834/3717777626,v1:172.21.15.111:6835/3717777626] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{1:24469} state up:active seq 16 addr [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{1:24421} state up:standby-replay seq 1 addr [v2:172.21.15.196:6832/2394579385,v1:172.21.15.196:6833/2394579385] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{2:24454} state up:active seq 18 addr [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{2:24416} state up:standby-replay seq 1 addr [v2:172.21.15.111:6832/3618359356,v1:172.21.15.111:6833/3618359356] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{3:24452} state up:active seq 17 addr [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{3:14580} state up:standby-replay seq 1 addr [v2:172.21.15.19:6840/733319067,v1:172.21.15.19:6841/733319067] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{4:24448} state up:active seq 19 addr [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{4:14574} state up:standby-replay seq 1 addr [v2:172.21.15.19:6838/821554910,v1:172.21.15.19:6839/821554910] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.070 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:45.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-09-19T04:06:45.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.19:6834/243568683,v1:172.21.15.19:6835/243568683] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.19:6836/2162743100,v1:172.21.15.19:6837/2162743100] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T04:06:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-19T04:06:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9ffc0a3270 msgr2=0x7f9ffc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9ffc0a3270 0x7f9ffc0a5730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa010101b30 tx=0x7fa00c00a560 comp rx=0 tx=0).stop 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 msgr2=0x7fa01019f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 0x7fa01019f470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa008005e30 tx=0x7fa008031ee0 comp rx=0 tx=0).stop 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 shutdown_connections 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f9ffc0a3270 0x7f9ffc0a5730 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa010102690 0x7fa01019f470 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa010101cd0 0x7fa0101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 --2- 172.21.15.19:0/2173677755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa010100ad0 0x7fa010197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 >> 172.21.15.19:0/2173677755 conn(0x7fa0100fc280 msgr2=0x7fa010105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 shutdown_connections 2024-09-19T04:06:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:45.066+0000 7fa017f60640 1 -- 172.21.15.19:0/2173677755 wait complete. 2024-09-19T04:06:45.847 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T04:06:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:46.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:45 smithi019 ceph-mon[30938]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-19T04:06:46.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:45 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2173677755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:06:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:45 smithi196 ceph-mon[35731]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-19T04:06:46.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:45 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2173677755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:06:46.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:45 smithi111 ceph-mon[35365]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-19T04:06:46.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:45 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2173677755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T04:06:46.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 -- 172.21.15.19:0/2061819125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220107900 msgr2=0x7fe220109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:46.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 --2- 172.21.15.19:0/2061819125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220107900 0x7fe220109d50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe218009ca0 tx=0x7fe21802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 -- 172.21.15.19:0/2061819125 shutdown_connections 2024-09-19T04:06:46.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 --2- 172.21.15.19:0/2061819125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220107900 0x7fe220109d50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 --2- 172.21.15.19:0/2061819125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe220104fd0 0x7fe2201073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 --2- 172.21.15.19:0/2061819125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2201026a0 0x7fe220104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 -- 172.21.15.19:0/2061819125 >> 172.21.15.19:0/2061819125 conn(0x7fe2200fc2a0 msgr2=0x7fe2200fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 -- 172.21.15.19:0/2061819125 shutdown_connections 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 -- 172.21.15.19:0/2061819125 wait complete. 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.607+0000 7fe22555e640 1 Processor -- start 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe22555e640 1 -- start start 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe22555e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2201026a0 0x7fe220102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe22555e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220104fd0 0x7fe220100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe22555e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe220107900 0x7fe220100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe22555e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010d8c0 con 0x7fe220104fd0 2024-09-19T04:06:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe22555e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010d740 con 0x7fe2201026a0 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe22555e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010da40 con 0x7fe220107900 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220104fd0 0x7fe220100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe21effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2201026a0 0x7fe220102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220104fd0 0x7fe220100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57798/0 (socket says 172.21.15.19:57798) 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 -- 172.21.15.19:0/3741551847 learned_addr learned my addr 172.21.15.19:0/3741551847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe21f7fe640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe220107900 0x7fe220100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 -- 172.21.15.19:0/3741551847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe220107900 msgr2=0x7fe220100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe220107900 0x7fe220100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 -- 172.21.15.19:0/3741551847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2201026a0 msgr2=0x7fe220102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2201026a0 0x7fe220102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 -- 172.21.15.19:0/3741551847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe220101450 con 0x7fe220104fd0 2024-09-19T04:06:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.608+0000 7fe217fff640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220104fd0 0x7fe220100650 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe20800d740 tx=0x7fe20800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.609+0000 7fe21cff9640 1 -- 172.21.15.19:0/3741551847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208017070 con 0x7fe220104fd0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.609+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2201a1ff0 con 0x7fe220104fd0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.609+0000 7fe21cff9640 1 -- 172.21.15.19:0/3741551847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe208009820 con 0x7fe220104fd0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.609+0000 7fe21cff9640 1 -- 172.21.15.19:0/3741551847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208004e00 con 0x7fe220104fd0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.609+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a2450 con 0x7fe220104fd0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.610+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2200fdd90 con 0x7fe220104fd0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.611+0000 7fe21cff9640 1 -- 172.21.15.19:0/3741551847 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe2080099d0 con 0x7fe220104fd0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.611+0000 7fe21cff9640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe20c0a3240 0x7fe20c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.612+0000 7fe21effd640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe20c0a3240 0x7fe20c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.612+0000 7fe21cff9640 1 -- 172.21.15.19:0/3741551847 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fe2080cf5b0 con 0x7fe220104fd0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.612+0000 7fe21effd640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe20c0a3240 0x7fe20c0a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe21000abe0 tx=0x7fe21000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.616+0000 7fe21cff9640 1 -- 172.21.15.19:0/3741551847 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe20809bf30 con 0x7fe220104fd0 2024-09-19T04:06:46.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.793+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe220062df0 con 0x7fe220104fd0 2024-09-19T04:06:46.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.794+0000 7fe21cff9640 1 -- 172.21.15.19:0/3741551847 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5647 (secure 0 0 0) 0x7fe20809b830 con 0x7fe220104fd0 2024-09-19T04:06:46.796 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 93 2024-09-19T04:06:46.796 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T04:06:46.796 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-09-19T03:56:42.596342+0000 2024-09-19T04:06:46.796 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-09-19T04:06:42.902449+0000 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 27 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T04:06:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/2401652913,v1:172.21.15.19:6803/2401652913] [v2:172.21.15.19:6804/2401652913,v1:172.21.15.19:6805/2401652913] exists,up ab98d168-5199-4c3f-a220-00c832addff2 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/3201221054,v1:172.21.15.111:6801/3201221054] [v2:172.21.15.111:6802/3201221054,v1:172.21.15.111:6803/3201221054] exists,up 66cb950f-6aee-4140-8411-0e9deaf594fc 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/4085056128,v1:172.21.15.196:6801/4085056128] [v2:172.21.15.196:6802/4085056128,v1:172.21.15.196:6803/4085056128] exists,up d2ad4ffa-1018-4519-96f8-a3ef93a779bb 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 25 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/3319817620,v1:172.21.15.19:6811/3319817620] [v2:172.21.15.19:6812/3319817620,v1:172.21.15.19:6813/3319817620] exists,up 1c13f30d-2850-46f4-a0d5-9f8d96b699ce 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 32 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/942057165,v1:172.21.15.111:6809/942057165] [v2:172.21.15.111:6810/942057165,v1:172.21.15.111:6811/942057165] exists,up fa74ad7b-8602-4087-aee4-c6e9632864c5 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 37 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/919577999,v1:172.21.15.196:6809/919577999] [v2:172.21.15.196:6810/919577999,v1:172.21.15.196:6811/919577999] exists,up 14c144ac-b587-4412-8aae-b0aef6058124 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 45 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/3521642658,v1:172.21.15.19:6819/3521642658] [v2:172.21.15.19:6820/3521642658,v1:172.21.15.19:6821/3521642658] exists,up 531057c9-5241-4e2e-9f6a-5ce00fb4cd26 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 50 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/837241728,v1:172.21.15.111:6817/837241728] [v2:172.21.15.111:6818/837241728,v1:172.21.15.111:6819/837241728] exists,up 44f8f0de-ffc4-4b29-b40a-73b485a3c478 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 57 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/189709907,v1:172.21.15.196:6817/189709907] [v2:172.21.15.196:6818/189709907,v1:172.21.15.196:6819/189709907] exists,up 8cdff67f-fc25-43ca-99e0-cb2063ee063c 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 65 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/2115247228,v1:172.21.15.19:6827/2115247228] [v2:172.21.15.19:6828/2115247228,v1:172.21.15.19:6829/2115247228] exists,up 1f507340-9ceb-4005-9247-bb7fac27bdf3 2024-09-19T04:06:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 70 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6824/610848665,v1:172.21.15.111:6825/610848665] [v2:172.21.15.111:6826/610848665,v1:172.21.15.111:6827/610848665] exists,up 17727a7c-08c8-46ba-b9f9-e0704680e6ff 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 77 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/301665031,v1:172.21.15.196:6825/301665031] [v2:172.21.15.196:6826/301665031,v1:172.21.15.196:6827/301665031] exists,up f71692d8-81b5-439b-b1c2-8972209815cb 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.b [8,11,2,1] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.f [8,11] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.14 [2,4] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.0 [5,9] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.6 [3,6] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.13 [4,9] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.17 [4,7] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.21 [4,8,5,9] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.6 [9,1] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.8 [5,8] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.e [9,11] 2024-09-19T04:06:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.10 [9,8] 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1292798638 expires 2024-09-20T03:57:40.326582+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/2229567042 expires 2024-09-20T03:57:40.326582+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/2229567042 expires 2024-09-20T03:57:40.326582+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/2828287970 expires 2024-09-20T03:57:40.326582+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/569607576 expires 2024-09-20T03:57:15.012251+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/748113766 expires 2024-09-20T03:57:15.012251+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3673052449 expires 2024-09-20T03:57:15.012251+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3967546781 expires 2024-09-20T03:57:40.326582+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/3897870084 expires 2024-09-20T03:57:15.012251+0000 2024-09-19T04:06:46.800 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/3897870084 expires 2024-09-20T03:57:15.012251+0000 2024-09-19T04:06:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe20c0a3240 msgr2=0x7fe20c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe20c0a3240 0x7fe20c0a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe21000abe0 tx=0x7fe21000a2d0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220104fd0 msgr2=0x7fe220100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220104fd0 0x7fe220100650 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe20800d740 tx=0x7fe20800db40 comp rx=0 tx=0).stop 2024-09-19T04:06:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 shutdown_connections 2024-09-19T04:06:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe20c0a3240 0x7fe20c0a5700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe220107900 0x7fe220100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe220104fd0 0x7fe220100650 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 --2- 172.21.15.19:0/3741551847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2201026a0 0x7fe220102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 >> 172.21.15.19:0/3741551847 conn(0x7fe2200fc2a0 msgr2=0x7fe220102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 shutdown_connections 2024-09-19T04:06:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:46.796+0000 7fe22555e640 1 -- 172.21.15.19:0/3741551847 wait complete. 2024-09-19T04:06:47.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:46 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3741551847' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:06:47.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:46 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3741551847' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:06:47.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:46 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3741551847' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T04:06:47.458 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T04:06:47.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:06:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:48.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:47 smithi019 ceph-mon[30938]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-19T04:06:48.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:47 smithi196 ceph-mon[35731]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 -- 172.21.15.19:0/466547159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 msgr2=0x7fd71810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 --2- 172.21.15.19:0/466547159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 0x7fd71810da40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd710009ca0 tx=0x7fd71002a950 comp rx=0 tx=0).stop 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 -- 172.21.15.19:0/466547159 shutdown_connections 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 --2- 172.21.15.19:0/466547159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 0x7fd71810da40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 --2- 172.21.15.19:0/466547159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd718102570 0x7fd7181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 --2- 172.21.15.19:0/466547159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718100e20 0x7fd718101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 -- 172.21.15.19:0/466547159 >> 172.21.15.19:0/466547159 conn(0x7fd7180fc7d0 msgr2=0x7fd7180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 -- 172.21.15.19:0/466547159 shutdown_connections 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 -- 172.21.15.19:0/466547159 wait complete. 2024-09-19T04:06:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.246+0000 7fd71fef8640 1 Processor -- start 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71fef8640 1 -- start start 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71fef8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd718100e20 0x7fd7180fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71fef8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718102570 0x7fd7180fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71fef8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 0x7fd7180fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71fef8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd718110f80 con 0x7fd718102ef0 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71fef8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd718110e00 con 0x7fd718100e20 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71fef8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd718111100 con 0x7fd718102570 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 0x7fd7180fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 0x7fd7180fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57814/0 (socket says 172.21.15.19:57814) 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71dc94640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd718100e20 0x7fd7180fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 -- 172.21.15.19:0/1898722905 learned_addr learned my addr 172.21.15.19:0/1898722905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71d493640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718102570 0x7fd7180fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 -- 172.21.15.19:0/1898722905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718102570 msgr2=0x7fd7180fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718102570 0x7fd7180fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 -- 172.21.15.19:0/1898722905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd718100e20 msgr2=0x7fd7180fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd718100e20 0x7fd7180fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 -- 172.21.15.19:0/1898722905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7181a6890 con 0x7fd718102ef0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.247+0000 7fd71e495640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 0x7fd7180fd700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd71002ae70 tx=0x7fd71002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.248+0000 7fd70effd640 1 -- 172.21.15.19:0/1898722905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710034040 con 0x7fd718102ef0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.248+0000 7fd70effd640 1 -- 172.21.15.19:0/1898722905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd710030920 con 0x7fd718102ef0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.248+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7181a6b20 con 0x7fd718102ef0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.248+0000 7fd70effd640 1 -- 172.21.15.19:0/1898722905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100047f0 con 0x7fd718102ef0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.248+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7181a7000 con 0x7fd718102ef0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.249+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd718101200 con 0x7fd718102ef0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.250+0000 7fd70effd640 1 -- 172.21.15.19:0/1898722905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd710037020 con 0x7fd718102ef0 2024-09-19T04:06:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.250+0000 7fd70effd640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7040a3240 0x7fd7040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.251+0000 7fd71dc94640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7040a3240 0x7fd7040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.251+0000 7fd70effd640 1 -- 172.21.15.19:0/1898722905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fd7100f0b90 con 0x7fd718102ef0 2024-09-19T04:06:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.251+0000 7fd71dc94640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7040a3240 0x7fd7040a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd71400abe0 tx=0x7fd71400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.254+0000 7fd70effd640 1 -- 172.21.15.19:0/1898722905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd7100bd510 con 0x7fd718102ef0 2024-09-19T04:06:48.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:47 smithi111 ceph-mon[35365]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-19T04:06:48.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.465+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd71810a0a0 con 0x7fd718102ef0 2024-09-19T04:06:48.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.465+0000 7fd70effd640 1 -- 172.21.15.19:0/1898722905 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11980 (secure 0 0 0) 0x7fd7100bce10 con 0x7fd718102ef0 2024-09-19T04:06:48.468 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:48.468 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-19T04:05:23:602377+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:05:23.279052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24454,24452,24448]},"id":1}]} 2024-09-19T04:06:48.468 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-19T04:06:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7040a3240 msgr2=0x7fd7040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7040a3240 0x7fd7040a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd71400abe0 tx=0x7fd71400a2d0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 msgr2=0x7fd7180fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 0x7fd7180fd700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd71002ae70 tx=0x7fd71002f5d0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 shutdown_connections 2024-09-19T04:06:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7040a3240 0x7fd7040a5700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd718102ef0 0x7fd7180fd700 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718102570 0x7fd7180fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 --2- 172.21.15.19:0/1898722905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd718100e20 0x7fd7180fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 >> 172.21.15.19:0/1898722905 conn(0x7fd7180fc7d0 msgr2=0x7fd71806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 shutdown_connections 2024-09-19T04:06:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:48.468+0000 7fd71fef8640 1 -- 172.21.15.19:0/1898722905 wait complete. 2024-09-19T04:06:49.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:06:49.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:48 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1898722905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:49.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:48 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1898722905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:49.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:48 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1898722905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:49.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 -- 172.21.15.19:0/2655191779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800101420 msgr2=0x7f280010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:49.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 --2- 172.21.15.19:0/2655191779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800101420 0x7f280010d950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009cd0 tx=0x7f27f802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 -- 172.21.15.19:0/2655191779 shutdown_connections 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 --2- 172.21.15.19:0/2655191779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800101420 0x7f280010d950 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 --2- 172.21.15.19:0/2655191779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2800100a80 0x7f2800100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 --2- 172.21.15.19:0/2655191779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800106a80 0x7f2800106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 -- 172.21.15.19:0/2655191779 >> 172.21.15.19:0/2655191779 conn(0x7f28000fc7a0 msgr2=0x7f28000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 -- 172.21.15.19:0/2655191779 shutdown_connections 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.821+0000 7f28083a7640 1 -- 172.21.15.19:0/2655191779 wait complete. 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f28083a7640 1 Processor -- start 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f28083a7640 1 -- start start 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f28083a7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800100a80 0x7f280010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f28083a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2800101420 0x7f280010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f28083a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800106a80 0x7f280010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:49.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f28083a7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2800111fa0 con 0x7f2800106a80 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f28083a7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2800111e20 con 0x7f2800100a80 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f28083a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2800112120 con 0x7f2800101420 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806944640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800106a80 0x7f280010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806944640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800106a80 0x7f280010ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57836/0 (socket says 172.21.15.19:57836) 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806944640 1 -- 172.21.15.19:0/749987009 learned_addr learned my addr 172.21.15.19:0/749987009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806143640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800100a80 0x7f280010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806944640 1 -- 172.21.15.19:0/749987009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2800101420 msgr2=0x7f280010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806944640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2800101420 0x7f280010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806944640 1 -- 172.21.15.19:0/749987009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800100a80 msgr2=0x7f280010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806944640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800100a80 0x7f280010e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.822+0000 7f2806944640 1 -- 172.21.15.19:0/749987009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28001a68c0 con 0x7f2800106a80 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.823+0000 7f2806944640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800106a80 0x7f280010ee60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006010 tx=0x7f27f8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:49.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.823+0000 7f27f77fe640 1 -- 172.21.15.19:0/749987009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8004450 con 0x7f2800106a80 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.823+0000 7f27f77fe640 1 -- 172.21.15.19:0/749987009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f8003e10 con 0x7f2800106a80 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.823+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28001a6b50 con 0x7f2800106a80 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.823+0000 7f27f77fe640 1 -- 172.21.15.19:0/749987009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8046480 con 0x7f2800106a80 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.823+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28001a7090 con 0x7f2800106a80 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.824+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2800100ee0 con 0x7f2800106a80 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.825+0000 7f27f77fe640 1 -- 172.21.15.19:0/749987009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f27f80045f0 con 0x7f2800106a80 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.826+0000 7f27f77fe640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f27ec0a3240 0x7f27ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.826+0000 7f2806143640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f27ec0a3240 0x7f27ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.826+0000 7f27f77fe640 1 -- 172.21.15.19:0/749987009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f27f802e030 con 0x7f2800106a80 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.827+0000 7f2806143640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f27ec0a3240 0x7f27ec0a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00abe0 tx=0x7f27fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:49.829+0000 7f27f77fe640 1 -- 172.21.15.19:0/749987009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27f804b060 con 0x7f2800106a80 2024-09-19T04:06:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.046+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28001a7360 con 0x7f2800106a80 2024-09-19T04:06:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:49 smithi019 ceph-mon[30938]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 639 B/s wr, 12 op/s 2024-09-19T04:06:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.047+0000 7f27f77fe640 1 -- 172.21.15.19:0/749987009 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11980 (secure 0 0 0) 0x7f27f80be130 con 0x7f2800106a80 2024-09-19T04:06:50.050 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:50.050 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-19T04:05:23:602377+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:05:23.279052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24454,24452,24448]},"id":1}]} 2024-09-19T04:06:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-19T04:06:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f27ec0a3240 msgr2=0x7f27ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f27ec0a3240 0x7f27ec0a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00abe0 tx=0x7f27fc00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800106a80 msgr2=0x7f280010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800106a80 0x7f280010ee60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006010 tx=0x7f27f8003c20 comp rx=0 tx=0).stop 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 shutdown_connections 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f27ec0a3240 0x7f27ec0a5700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2800106a80 0x7f280010ee60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2800101420 0x7f280010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 --2- 172.21.15.19:0/749987009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800100a80 0x7f280010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 >> 172.21.15.19:0/749987009 conn(0x7f28000fc7a0 msgr2=0x7f28000fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 shutdown_connections 2024-09-19T04:06:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:50.049+0000 7f28083a7640 1 -- 172.21.15.19:0/749987009 wait complete. 2024-09-19T04:06:50.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:49 smithi196 ceph-mon[35731]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 639 B/s wr, 12 op/s 2024-09-19T04:06:50.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:49 smithi111 ceph-mon[35365]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 639 B/s wr, 12 op/s 2024-09-19T04:06:50.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:06:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:51.138 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:50 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/749987009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:51.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:50 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/749987009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:51.306 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:50 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/749987009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:06:51.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.471+0000 7ff5d45bf640 1 -- 172.21.15.19:0/2184360422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc103660 msgr2=0x7ff5cc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:51.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.471+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/2184360422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc103660 0x7ff5cc10bb50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009900 tx=0x7ff5bc02a480 comp rx=0 tx=0).stop 2024-09-19T04:06:51.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.471+0000 7ff5d45bf640 1 -- 172.21.15.19:0/2184360422 shutdown_connections 2024-09-19T04:06:51.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.471+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/2184360422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc10c170 0x7ff5cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.471+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/2184360422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc103660 0x7ff5cc10bb50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.471+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/2184360422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc102d40 0x7ff5cc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.471+0000 7ff5d45bf640 1 -- 172.21.15.19:0/2184360422 >> 172.21.15.19:0/2184360422 conn(0x7ff5cc0fc7a0 msgr2=0x7ff5cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 -- 172.21.15.19:0/2184360422 shutdown_connections 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 -- 172.21.15.19:0/2184360422 wait complete. 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 Processor -- start 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 -- start start 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc103660 0x7ff5cc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc10c170 0x7ff5cc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111fb0 con 0x7ff5cc102d40 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111e30 con 0x7ff5cc10c170 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d45bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc112130 con 0x7ff5cc103660 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d235b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.472+0000 7ff5d235b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46402/0 (socket says 172.21.15.19:46402) 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d235b640 1 -- 172.21.15.19:0/1021956709 learned_addr learned my addr 172.21.15.19:0/1021956709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d2b5c640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc10c170 0x7ff5cc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d235b640 1 -- 172.21.15.19:0/1021956709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc103660 msgr2=0x7ff5cc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d1b5a640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc103660 0x7ff5cc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d235b640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc103660 0x7ff5cc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d235b640 1 -- 172.21.15.19:0/1021956709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc10c170 msgr2=0x7ff5cc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:51.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d235b640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc10c170 0x7ff5cc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d235b640 1 -- 172.21.15.19:0/1021956709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc1a6960 con 0x7ff5cc102d40 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d235b640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1009a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800d950 tx=0x7ff5c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5c37fe640 1 -- 172.21.15.19:0/1021956709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8017030 con 0x7ff5cc102d40 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5c37fe640 1 -- 172.21.15.19:0/1021956709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8004020 con 0x7ff5cc102d40 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc1a6c50 con 0x7ff5cc102d40 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5c37fe640 1 -- 172.21.15.19:0/1021956709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80047a0 con 0x7ff5cc102d40 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.473+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1a7190 con 0x7ff5cc102d40 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.474+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5cc103120 con 0x7ff5cc102d40 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.475+0000 7ff5c37fe640 1 -- 172.21.15.19:0/1021956709 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff5c80041d0 con 0x7ff5cc102d40 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.476+0000 7ff5c37fe640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5b80a32c0 0x7ff5b80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.476+0000 7ff5c37fe640 1 -- 172.21.15.19:0/1021956709 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7ff5c80cf610 con 0x7ff5cc102d40 2024-09-19T04:06:51.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.476+0000 7ff5d1b5a640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5b80a32c0 0x7ff5b80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:51.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.477+0000 7ff5d1b5a640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5b80a32c0 0x7ff5b80a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0096c0 tx=0x7ff5bc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:51.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.480+0000 7ff5c37fe640 1 -- 172.21.15.19:0/1021956709 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5c809bf90 con 0x7ff5cc102d40 2024-09-19T04:06:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.658+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5cc106c50 con 0x7ff5cc102d40 2024-09-19T04:06:51.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.660+0000 7ff5c37fe640 1 -- 172.21.15.19:0/1021956709 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24842 (secure 0 0 0) 0x7ff5c809b890 con 0x7ff5cc102d40 2024-09-19T04:06:51.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:06:51.663 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":93,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:06:42.902449+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:06:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5b80a32c0 msgr2=0x7ff5b80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5b80a32c0 0x7ff5b80a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0096c0 tx=0x7ff5bc000bc0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc102d40 msgr2=0x7ff5cc1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1009a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800d950 tx=0x7ff5c800de10 comp rx=0 tx=0).stop 2024-09-19T04:06:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 shutdown_connections 2024-09-19T04:06:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ff5b80a32c0 0x7ff5b80a5780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5cc10c170 0x7ff5cc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc103660 0x7ff5cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 --2- 172.21.15.19:0/1021956709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5cc102d40 0x7ff5cc1009a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 >> 172.21.15.19:0/1021956709 conn(0x7ff5cc0fc7a0 msgr2=0x7ff5cc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 shutdown_connections 2024-09-19T04:06:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:51.663+0000 7ff5d45bf640 1 -- 172.21.15.19:0/1021956709 wait complete. 2024-09-19T04:06:52.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:51 smithi196 ceph-mon[35731]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 479 B/s wr, 11 op/s 2024-09-19T04:06:52.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:51 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1021956709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:06:52.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:51 smithi019 ceph-mon[30938]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 479 B/s wr, 11 op/s 2024-09-19T04:06:52.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:51 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1021956709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:06:52.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:04:40.077653+0000', 'modified': '2024-09-19T04:05:23.279052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24469, 'mds_2': 24454, 'mds_3': 24452, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14574': {'gid': 14574, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/821554910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 821554910}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 821554910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14580': {'gid': 14580, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/733319067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 733319067}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 733319067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6833/3618359356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 3618359356}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 3618359356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/2394579385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 2394579385}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 2394579385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/3717777626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 3717777626}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 3717777626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.196:6835/1584544131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1584544131}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1584544131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.111:6837/1580295791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 1580295791}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 1580295791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.196:6837/1041250110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1041250110}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1041250110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.196:6839/1173261295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1173261295}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1173261295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.111:6839/528342006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 528342006}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 528342006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24476, 24469, 24454, 24452, 24448]} 2024-09-19T04:06:52.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T04:06:52.364 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T04:06:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:51 smithi111 ceph-mon[35365]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 479 B/s wr, 11 op/s 2024-09-19T04:06:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:51 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1021956709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:06:52.627 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.130+0000 7f1515292640 1 -- 172.21.15.19:0/2716257219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510101de0 msgr2=0x7f15101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.130+0000 7f1515292640 1 --2- 172.21.15.19:0/2716257219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510101de0 0x7f15101021c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1500009900 tx=0x7f150002a410 comp rx=0 tx=0).stop 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.130+0000 7f1515292640 1 -- 172.21.15.19:0/2716257219 shutdown_connections 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.130+0000 7f1515292640 1 --2- 172.21.15.19:0/2716257219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510106f10 0x7f151010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.130+0000 7f1515292640 1 --2- 172.21.15.19:0/2716257219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1510102790 0x7f15101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.130+0000 7f1515292640 1 --2- 172.21.15.19:0/2716257219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510101de0 0x7f15101021c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.130+0000 7f1515292640 1 -- 172.21.15.19:0/2716257219 >> 172.21.15.19:0/2716257219 conn(0x7f15100fd8c0 msgr2=0x7f15100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 -- 172.21.15.19:0/2716257219 shutdown_connections 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 -- 172.21.15.19:0/2716257219 wait complete. 2024-09-19T04:06:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 Processor -- start 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 -- start start 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510101de0 0x7f15101983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510102790 0x7f15101988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1510106f10 0x7f151019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1510111f70 con 0x7f1510102790 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1510111df0 con 0x7f1510106f10 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1515292640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15101120f0 con 0x7f1510101de0 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.131+0000 7f1507fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510102790 0x7f15101988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1507fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510102790 0x7f15101988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46426/0 (socket says 172.21.15.19:46426) 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1507fff640 1 -- 172.21.15.19:0/2324087026 learned_addr learned my addr 172.21.15.19:0/2324087026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f150f7fe640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1510106f10 0x7f151019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f150effd640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510101de0 0x7f15101983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1507fff640 1 -- 172.21.15.19:0/2324087026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510101de0 msgr2=0x7f15101983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1507fff640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510101de0 0x7f15101983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1507fff640 1 -- 172.21.15.19:0/2324087026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1510106f10 msgr2=0x7f151019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1507fff640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1510106f10 0x7f151019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1507fff640 1 -- 172.21.15.19:0/2324087026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151019d360 con 0x7f1510102790 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f150f7fe640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1510106f10 0x7f151019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1507fff640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510102790 0x7f15101988f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c890 tx=0x7f14f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f150cff9640 1 -- 172.21.15.19:0/2324087026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8004220 con 0x7f1510102790 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f150cff9640 1 -- 172.21.15.19:0/2324087026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f80043c0 con 0x7f1510102790 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f150cff9640 1 -- 172.21.15.19:0/2324087026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f80049e0 con 0x7f1510102790 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1515292640 1 -- 172.21.15.19:0/2324087026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151019d650 con 0x7f1510102790 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.132+0000 7f1515292640 1 -- 172.21.15.19:0/2324087026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15101a4f30 con 0x7f1510102790 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f150cff9640 1 -- 172.21.15.19:0/2324087026 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f14f80079b0 con 0x7f1510102790 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f151766b740 1 Processor -- start 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f151766b740 1 -- start start 2024-09-19T04:06:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f151766b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184f98f0 0x562b184f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f151766b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b18508a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f151766b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b18508fc0 0x562b1850b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f151766b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b18460090 con 0x562b184f98f0 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f151766b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562b18407a70 con 0x562b18508fc0 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.135+0000 7f151766b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b1845c050 con 0x562b184fa210 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184f98f0 0x562b184f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184f98f0 0x562b184f9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46442/0 (socket says 172.21.15.19:46442) 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 -- 172.21.15.19:0/809094688 learned_addr learned my addr 172.21.15.19:0/809094688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150cff9640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f14fc0a32c0 0x7f14fc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150effd640 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b18508a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f1507fff640 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b18508fc0 0x562b1850b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f14fc0a32c0 0x7f14fc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150cff9640 1 -- 172.21.15.19:0/2324087026 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f14f80cfde0 con 0x7f1510102790 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 -- 172.21.15.19:0/809094688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 msgr2=0x562b18508a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b18508a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 -- 172.21.15.19:0/809094688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b18508fc0 msgr2=0x562b1850b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b18508fc0 0x562b1850b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 -- 172.21.15.19:0/809094688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b18414130 con 0x562b184f98f0 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f1507fff640 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b18508fc0 0x562b1850b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f150f7fe640 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184f98f0 0x562b184f9cd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f15080149a0 tx=0x7f1508014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27072aeaf3095443 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f15057fa640 1 -- 172.21.15.19:0/809094688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508021070 con 0x562b184f98f0 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f15057fa640 1 -- 172.21.15.19:0/809094688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1508013460 con 0x562b184f98f0 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.136+0000 7f15057fa640 1 -- 172.21.15.19:0/809094688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150800f5d0 con 0x562b184f98f0 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 -- 172.21.15.19:0/809094688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184f98f0 msgr2=0x562b184f9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f150f7fe640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f14fc0a32c0 0x7f14fc0a5780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f150801e000 tx=0x7f1508019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184f98f0 0x562b184f9cd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f15080149a0 tx=0x7f1508014e60 comp rx=0 tx=0).stop 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 -- 172.21.15.19:0/809094688 shutdown_connections 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b18508fc0 0x562b1850b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b18508a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 --2- 172.21.15.19:0/809094688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184f98f0 0x562b184f9cd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 -- 172.21.15.19:0/809094688 >> 172.21.15.19:0/809094688 conn(0x562b184f5f60 msgr2=0x562b184f6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 -- 172.21.15.19:0/809094688 shutdown_connections 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.137+0000 7f151766b740 1 -- 172.21.15.19:0/809094688 wait complete. 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 1 -- start start 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 1 Processor -- start 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b185088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184fa210 0x562b18505bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b18508fc0 0x562b18506130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b1850bb30 con 0x562b184fa210 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562b1841af20 con 0x562b184f98f0 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b1850bcb0 con 0x562b18508fc0 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f151766b740 10 client.0 _ll_register_callbacks cb 0x562b183e4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f1507fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184fa210 0x562b18505bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f1507fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184fa210 0x562b18505bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46450/0 (socket says 172.21.15.19:46450) 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f1507fff640 1 -- 172.21.15.19:0/3558218405 learned_addr learned my addr 172.21.15.19:0/3558218405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f150effd640 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b185088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f150f7fe640 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b18508fc0 0x562b18506130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f1507fff640 1 -- 172.21.15.19:0/3558218405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b18508fc0 msgr2=0x562b18506130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f1507fff640 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b18508fc0 0x562b18506130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f1507fff640 1 -- 172.21.15.19:0/3558218405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 msgr2=0x562b185088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f1507fff640 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b185088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.138+0000 7f1507fff640 1 -- 172.21.15.19:0/3558218405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b181d12b0 con 0x562b184fa210 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f1507fff640 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184fa210 0x562b18505bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f14f80989d0 tx=0x7f14f8066e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa7a486afa7689e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f14f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f14f67fc640 1 -- 172.21.15.19:0/3558218405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f801d050 con 0x562b184fa210 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f14f67fc640 1 -- 172.21.15.19:0/3558218405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f14f80cd5f0 con 0x562b184fa210 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 1 -- 172.21.15.19:0/3558218405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1508013460 con 0x562b184fa210 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f14f67fc640 1 -- 172.21.15.19:0/3558218405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14f809c000 con 0x562b184fa210 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 10 client.15183 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 1 -- 172.21.15.19:0/3558218405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b185174e0 con 0x562b184fa210 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f14f67fc640 1 -- 172.21.15.19:0/3558218405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1508013460 con 0x562b184fa210 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f14f67fc640 1 -- 172.21.15.19:0/3558218405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f14f806bce0 con 0x562b184fa210 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 10 client.15183 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 10 client.15183 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 20 client.15183 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x562b18519ab0 0x562b1851af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 4 client.15183 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.139+0000 7f151766b740 1 -- 172.21.15.19:0/3558218405 --> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b1846d600 con 0x562b18519ab0 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.140+0000 7f150effd640 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x562b18519ab0 0x562b1851af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.140+0000 7f150effd640 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x562b18519ab0 0x562b1851af20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a4bee93150863157 server_cookie=6e105f65b18e3cfb in_seq=0 out_seq=0 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.140+0000 7f14f67fc640 10 client.15183 ms_handle_connect on v2:172.21.15.196:6834/1584544131 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f14f67fc640 1 -- 172.21.15.19:0/3558218405 <== mds.4 v2:172.21.15.196:6834/1584544131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b1846d600 con 0x562b18519ab0 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f14f67fc640 10 client.15183 handle_command_reply: tid=0 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 client.15183 shutdown 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 -- 172.21.15.19:0/3558218405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184fa210 msgr2=0x562b18505bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184fa210 0x562b18505bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f14f80989d0 tx=0x7f14f8066e30 comp rx=0 tx=0).stop 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 -- 172.21.15.19:0/3558218405 shutdown_connections 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b18508fc0 0x562b18506130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x562b18519ab0 0x562b1851af20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b184fa210 0x562b18505bf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 --2- 172.21.15.19:0/3558218405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b185088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f14f67fc640 0 client.15183 ms_handle_reset on v2:172.21.15.196:6834/1584544131 2024-09-19T04:06:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 -- 172.21.15.19:0/3558218405 >> 172.21.15.19:0/3558218405 conn(0x562b184f5f60 msgr2=0x562b184fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 -- 172.21.15.19:0/3558218405 shutdown_connections 2024-09-19T04:06:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 1 -- 172.21.15.19:0/3558218405 wait complete. 2024-09-19T04:06:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.141+0000 7f151766b740 20 client.15183 trim_cache size 0 max 16384 2024-09-19T04:06:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.159+0000 7f151766b740 1 Processor -- start 2024-09-19T04:06:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.159+0000 7f151766b740 1 -- start start 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f151766b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b185054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f151766b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b185044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f151766b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b18504a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f151766b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b18519260 con 0x562b18508fc0 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f151766b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562b18519930 con 0x562b184f98f0 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f151766b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b1851a000 con 0x562b184fa210 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b18504a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b18504a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46458/0 (socket says 172.21.15.19:46458) 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 -- 172.21.15.19:0/704161920 learned_addr learned my addr 172.21.15.19:0/704161920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150effd640 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b185054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f1507fff640 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b185044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 -- 172.21.15.19:0/704161920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 msgr2=0x562b185044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b185044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 -- 172.21.15.19:0/704161920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 msgr2=0x562b185054b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b185054b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 -- 172.21.15.19:0/704161920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b185084c0 con 0x562b18508fc0 2024-09-19T04:06:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.160+0000 7f150f7fe640 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b18504a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1508005cc0 tx=0x7f1508025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23a28c747b1aac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f1504ff9640 1 -- 172.21.15.19:0/704161920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508021040 con 0x562b18508fc0 2024-09-19T04:06:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f1504ff9640 1 -- 172.21.15.19:0/704161920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1508014040 con 0x562b18508fc0 2024-09-19T04:06:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f1504ff9640 1 -- 172.21.15.19:0/704161920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508005270 con 0x562b18508fc0 2024-09-19T04:06:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f151766b740 1 -- 172.21.15.19:0/704161920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 msgr2=0x562b18504a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f151766b740 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b18504a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1508005cc0 tx=0x7f1508025000 comp rx=0 tx=0).stop 2024-09-19T04:06:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f151766b740 1 -- 172.21.15.19:0/704161920 shutdown_connections 2024-09-19T04:06:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f151766b740 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b18504a00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f151766b740 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b185044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f151766b740 1 --2- 172.21.15.19:0/704161920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b185054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.161+0000 7f151766b740 1 -- 172.21.15.19:0/704161920 >> 172.21.15.19:0/704161920 conn(0x562b184f5f60 msgr2=0x562b18505bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 -- 172.21.15.19:0/704161920 shutdown_connections 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 -- 172.21.15.19:0/704161920 wait complete. 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 -- start start 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 Processor -- start 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b18523360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b185238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b1851c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b1851ada0 con 0x562b18508fc0 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562b1851a6d0 con 0x562b184f98f0 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b1851b470 con 0x562b184fa210 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f151766b740 10 client.0 _ll_register_callbacks cb 0x7f14f8004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f150f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b1851c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f150f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b1851c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46464/0 (socket says 172.21.15.19:46464) 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.162+0000 7f150f7fe640 1 -- 172.21.15.19:0/1848121640 learned_addr learned my addr 172.21.15.19:0/1848121640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f150effd640 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b18523360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f1507fff640 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b185238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f150f7fe640 1 -- 172.21.15.19:0/1848121640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 msgr2=0x562b185238a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f150f7fe640 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b185238a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f150f7fe640 1 -- 172.21.15.19:0/1848121640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 msgr2=0x562b18523360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f150f7fe640 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b18523360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f150f7fe640 1 -- 172.21.15.19:0/1848121640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1851cfe0 con 0x562b18508fc0 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f150f7fe640 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b1851c7b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1508014840 tx=0x7f1508014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0623d1aa1c33a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f14f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f14f67fc640 1 -- 172.21.15.19:0/1848121640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508004db0 con 0x562b18508fc0 2024-09-19T04:06:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f14f67fc640 1 -- 172.21.15.19:0/1848121640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f1508034040 con 0x562b18508fc0 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f151766b740 1 -- 172.21.15.19:0/1848121640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1508014040 con 0x562b18508fc0 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f14f67fc640 1 -- 172.21.15.19:0/1848121640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1508043050 con 0x562b18508fc0 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f151766b740 10 client.15189 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f151766b740 1 -- 172.21.15.19:0/1848121640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b18521880 con 0x562b18508fc0 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.163+0000 7f14f67fc640 1 -- 172.21.15.19:0/1848121640 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1508014040 con 0x562b18508fc0 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f14f67fc640 1 -- 172.21.15.19:0/1848121640 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1508038430 con 0x562b18508fc0 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f151766b740 10 client.15189 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f151766b740 10 client.15189 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f151766b740 20 client.15189 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f151766b740 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x562b18588050 0x562b18586d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f151766b740 4 client.15189 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f151766b740 1 -- 172.21.15.19:0/1848121640 --> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] -- command(tid 0: {"prefix": "status"}) -- 0x562b18467b10 con 0x562b18588050 2024-09-19T04:06:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f150effd640 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x562b18588050 0x562b18586d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f150effd640 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x562b18588050 0x562b18586d60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef0de8ee383ae624 server_cookie=927cf6baca6364dc in_seq=0 out_seq=0 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.164+0000 7f14f67fc640 10 client.15189 ms_handle_connect on v2:172.21.15.196:6834/1584544131 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f14f67fc640 1 -- 172.21.15.19:0/1848121640 <== mds.4 v2:172.21.15.196:6834/1584544131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562b18467b10 con 0x562b18588050 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f14f67fc640 10 client.15189 handle_command_reply: tid=0 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 client.15189 shutdown 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 -- 172.21.15.19:0/1848121640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 msgr2=0x562b1851c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b1851c7b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1508014840 tx=0x7f1508014e60 comp rx=0 tx=0).stop 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 -- 172.21.15.19:0/1848121640 shutdown_connections 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x562b18588050 0x562b18586d60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b18508fc0 0x562b1851c7b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b184fa210 0x562b185238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 --2- 172.21.15.19:0/1848121640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b184f98f0 0x562b18523360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 -- 172.21.15.19:0/1848121640 >> 172.21.15.19:0/1848121640 conn(0x562b184f5f60 msgr2=0x562b18500a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f14f67fc640 0 client.15189 ms_handle_reset on v2:172.21.15.196:6834/1584544131 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 -- 172.21.15.19:0/1848121640 shutdown_connections 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 1 -- 172.21.15.19:0/1848121640 wait complete. 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.165+0000 7f151766b740 20 client.15189 trim_cache size 0 max 16384 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24448, 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:53.173 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:53.174 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:53.174 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 90.645923991999993, 2024-09-19T04:06:53.174 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:53.174 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:06:53.174 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:53.174 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 159.78652973199999 2024-09-19T04:06:53.174 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 -- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f14fc0a32c0 msgr2=0x7f14fc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f14fc0a32c0 0x7f14fc0a5780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f150801e000 tx=0x7f1508019000 comp rx=0 tx=0).stop 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 -- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510102790 msgr2=0x7f15101988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510102790 0x7f15101988f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c890 tx=0x7f14f800cd50 comp rx=0 tx=0).stop 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 -- 172.21.15.19:0/2324087026 shutdown_connections 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f14fc0a32c0 0x7f14fc0a5780 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1510106f10 0x7f151019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510102790 0x7f15101988f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 --2- 172.21.15.19:0/2324087026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1510101de0 0x7f15101983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 -- 172.21.15.19:0/2324087026 >> 172.21.15.19:0/2324087026 conn(0x7f15100fd8c0 msgr2=0x7f15100fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 -- 172.21.15.19:0/2324087026 shutdown_connections 2024-09-19T04:06:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:53.166+0000 7f1515292640 1 -- 172.21.15.19:0/2324087026 wait complete. 2024-09-19T04:06:53.910 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T04:06:54.169 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:54.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:53 smithi019 ceph-mon[30938]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-19T04:06:54.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:53 smithi196 ceph-mon[35731]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-19T04:06:54.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:53 smithi111 ceph-mon[35365]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-19T04:06:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 -- 172.21.15.19:0/3641903761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc106f10 msgr2=0x7facbc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 --2- 172.21.15.19:0/3641903761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc106f10 0x7facbc10ea90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7facb4009ca0 tx=0x7facb402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 -- 172.21.15.19:0/3641903761 shutdown_connections 2024-09-19T04:06:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 --2- 172.21.15.19:0/3641903761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc106f10 0x7facbc10ea90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 --2- 172.21.15.19:0/3641903761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facbc102790 0x7facbc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 --2- 172.21.15.19:0/3641903761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facbc101de0 0x7facbc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 -- 172.21.15.19:0/3641903761 >> 172.21.15.19:0/3641903761 conn(0x7facbc0fd8c0 msgr2=0x7facbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 -- 172.21.15.19:0/3641903761 shutdown_connections 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.655+0000 7facc3160640 1 -- 172.21.15.19:0/3641903761 wait complete. 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc3160640 1 Processor -- start 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc3160640 1 -- start start 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc3160640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc101de0 0x7facbc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc3160640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facbc102790 0x7facbc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc3160640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facbc106f10 0x7facbc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc3160640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7facbc111f70 con 0x7facbc101de0 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc3160640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7facbc111df0 con 0x7facbc106f10 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc3160640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facbc1120f0 con 0x7facbc102790 2024-09-19T04:06:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc0efc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc101de0 0x7facbc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc0efc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc101de0 0x7facbc1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46476/0 (socket says 172.21.15.19:46476) 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facc0efc640 1 -- 172.21.15.19:0/3674960853 learned_addr learned my addr 172.21.15.19:0/3674960853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.656+0000 7facbbfff640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facbc102790 0x7facbc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc0efc640 1 -- 172.21.15.19:0/3674960853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facbc102790 msgr2=0x7facbc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc0efc640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facbc102790 0x7facbc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc0efc640 1 -- 172.21.15.19:0/3674960853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facbc106f10 msgr2=0x7facbc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc16fd640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facbc106f10 0x7facbc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc0efc640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facbc106f10 0x7facbc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc0efc640 1 -- 172.21.15.19:0/3674960853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facbc19d480 con 0x7facbc101de0 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc0efc640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc101de0 0x7facbc1984d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7facac00c890 tx=0x7facac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facb9ffb640 1 -- 172.21.15.19:0/3674960853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac004220 con 0x7facbc101de0 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facb9ffb640 1 -- 172.21.15.19:0/3674960853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facac0043c0 con 0x7facbc101de0 2024-09-19T04:06:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc3160640 1 -- 172.21.15.19:0/3674960853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facbc19d770 con 0x7facbc101de0 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facb9ffb640 1 -- 172.21.15.19:0/3674960853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac0049e0 con 0x7facbc101de0 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.657+0000 7facc3160640 1 -- 172.21.15.19:0/3674960853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facbc1a5050 con 0x7facbc101de0 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.660+0000 7facb9ffb640 1 -- 172.21.15.19:0/3674960853 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7facac0079b0 con 0x7facbc101de0 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.660+0000 7facc5539740 1 Processor -- start 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.660+0000 7facc5539740 1 -- start start 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.660+0000 7facc5539740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e5564db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.660+0000 7facc5539740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc5539740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e5571fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc5539740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2e54cb090 con 0x55f2e55652f0 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc5539740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f2e5472a70 con 0x55f2e55649d0 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc5539740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2e54c7050 con 0x55f2e556fbb0 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46490/0 (socket says 172.21.15.19:46490) 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 -- 172.21.15.19:0/1169308482 learned_addr learned my addr 172.21.15.19:0/1169308482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc16fd640 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e5564db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facbbfff640 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e5571fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 -- 172.21.15.19:0/1169308482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 msgr2=0x55f2e5571fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e5571fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 -- 172.21.15.19:0/1169308482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 msgr2=0x55f2e5564db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e5564db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 -- 172.21.15.19:0/1169308482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2e547f130 con 0x55f2e55652f0 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facbbfff640 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e5571fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facb9ffb640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7faca80a3240 0x7faca80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc0efc640 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556f670 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7facac0d3eb0 tx=0x7facac0d8410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18c86c59b98b87d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facc16fd640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7faca80a3240 0x7faca80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facb9ffb640 1 -- 172.21.15.19:0/3674960853 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7facac0cfd10 con 0x7facbc101de0 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.661+0000 7facb2ffd640 1 -- 172.21.15.19:0/1169308482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac098230 con 0x55f2e55652f0 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facb2ffd640 1 -- 172.21.15.19:0/1169308482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facac0bc080 con 0x55f2e55652f0 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facb2ffd640 1 -- 172.21.15.19:0/1169308482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac09b8f0 con 0x55f2e55652f0 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 -- 172.21.15.19:0/1169308482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 msgr2=0x55f2e556f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc16fd640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7faca80a3240 0x7faca80a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facb4006010 tx=0x7facb4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556f670 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7facac0d3eb0 tx=0x7facac0d8410 comp rx=0 tx=0).stop 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 -- 172.21.15.19:0/1169308482 shutdown_connections 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e5571fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556f670 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 --2- 172.21.15.19:0/1169308482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e5564db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 -- 172.21.15.19:0/1169308482 >> 172.21.15.19:0/1169308482 conn(0x55f2e5560f60 msgr2=0x55f2e5561340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 -- 172.21.15.19:0/1169308482 shutdown_connections 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.662+0000 7facc5539740 1 -- 172.21.15.19:0/1169308482 wait complete. 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 1 -- start start 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 1 Processor -- start 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e556f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e556ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2e5572780 con 0x55f2e55652f0 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f2e5485f20 con 0x55f2e55649d0 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2e5572900 con 0x55f2e556fbb0 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc5539740 10 client.0 _ll_register_callbacks cb 0x55f2e544f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46500/0 (socket says 172.21.15.19:46500) 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facbbfff640 1 -- 172.21.15.19:0/3717169630 learned_addr learned my addr 172.21.15.19:0/3717169630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.663+0000 7facc16fd640 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e556ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facc0efc640 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e556f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facbbfff640 1 -- 172.21.15.19:0/3717169630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 msgr2=0x55f2e556ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facbbfff640 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e556ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facbbfff640 1 -- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 msgr2=0x55f2e556f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facbbfff640 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e556f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facbbfff640 1 -- 172.21.15.19:0/3717169630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2e523c2b0 con 0x55f2e55652f0 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facc0efc640 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e556f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facbbfff640 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556c7b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faca40126e0 tx=0x7faca4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f1a4939714d772 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7fac9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7fac9bfff640 1 -- 172.21.15.19:0/3717169630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca400f920 con 0x55f2e55652f0 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7fac9bfff640 1 -- 172.21.15.19:0/3717169630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7faca400fac0 con 0x55f2e55652f0 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facc5539740 1 -- 172.21.15.19:0/3717169630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facac0bc080 con 0x55f2e55652f0 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7fac9bfff640 1 -- 172.21.15.19:0/3717169630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faca400fcc0 con 0x55f2e55652f0 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facc5539740 10 client.15201 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7facc5539740 1 -- 172.21.15.19:0/3717169630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2e557e100 con 0x55f2e55652f0 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.664+0000 7fac9bfff640 1 -- 172.21.15.19:0/3717169630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facac0bc080 con 0x55f2e55652f0 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7fac9bfff640 1 -- 172.21.15.19:0/3717169630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7faca401c420 con 0x55f2e55652f0 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7facc5539740 10 client.15201 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7facc5539740 10 client.15201 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7facc5539740 20 client.15201 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7facc5539740 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55f2e55807c0 0x55f2e5581c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7facc5539740 4 client.15201 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7facc5539740 1 -- 172.21.15.19:0/3717169630 --> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2e54d8600 con 0x55f2e55807c0 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7facc0efc640 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55f2e55807c0 0x55f2e5581c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7facc0efc640 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55f2e55807c0 0x55f2e5581c30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ec958f32eb91497 server_cookie=30b9b8451efb4839 in_seq=0 out_seq=0 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.665+0000 7fac9bfff640 10 client.15201 ms_handle_connect on v2:172.21.15.111:6836/1580295791 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7fac9bfff640 1 -- 172.21.15.19:0/3717169630 <== mds.3 v2:172.21.15.111:6836/1580295791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2e54d8600 con 0x55f2e55807c0 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7fac9bfff640 10 client.15201 handle_command_reply: tid=0 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 client.15201 shutdown 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 -- 172.21.15.19:0/3717169630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 msgr2=0x55f2e556c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556c7b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faca40126e0 tx=0x7faca4012ba0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 -- 172.21.15.19:0/3717169630 shutdown_connections 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55f2e55807c0 0x55f2e5581c30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7fac9bfff640 0 client.15201 ms_handle_reset on v2:172.21.15.111:6836/1580295791 2024-09-19T04:06:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e556fbb0 0x55f2e556ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e556c7b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 --2- 172.21.15.19:0/3717169630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e556f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 -- 172.21.15.19:0/3717169630 >> 172.21.15.19:0/3717169630 conn(0x55f2e5560f60 msgr2=0x55f2e5566c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.666+0000 7facc5539740 1 -- 172.21.15.19:0/3717169630 shutdown_connections 2024-09-19T04:06:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.667+0000 7facc5539740 1 -- 172.21.15.19:0/3717169630 wait complete. 2024-09-19T04:06:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.667+0000 7facc5539740 20 client.15201 trim_cache size 0 max 16384 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc5539740 1 Processor -- start 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc5539740 1 -- start start 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc5539740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e5561340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc5539740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e5567560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc5539740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e5567aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc5539740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2e557ff70 con 0x55f2e55652f0 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc5539740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f2e5580640 con 0x55f2e55649d0 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc5539740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2e5580d10 con 0x55f2e5560f60 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e5567aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e5567aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46508/0 (socket says 172.21.15.19:46508) 2024-09-19T04:06:54.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 -- 172.21.15.19:0/3426659382 learned_addr learned my addr 172.21.15.19:0/3426659382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc0efc640 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e5561340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facbbfff640 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e5567560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 -- 172.21.15.19:0/3426659382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 msgr2=0x55f2e5561340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e5561340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 -- 172.21.15.19:0/3426659382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 msgr2=0x55f2e5567560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e5567560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 -- 172.21.15.19:0/3426659382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2e5572890 con 0x55f2e55652f0 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.685+0000 7facc16fd640 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e5567aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7facb4009ca0 tx=0x7facb4043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fdee2c8c4a663b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facb27fc640 1 -- 172.21.15.19:0/3426659382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4003d40 con 0x55f2e55652f0 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facb27fc640 1 -- 172.21.15.19:0/3426659382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb4036050 con 0x55f2e55652f0 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facb27fc640 1 -- 172.21.15.19:0/3426659382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb402fad0 con 0x55f2e55652f0 2024-09-19T04:06:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 -- 172.21.15.19:0/3426659382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 msgr2=0x55f2e5567aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e5567aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7facb4009ca0 tx=0x7facb4043000 comp rx=0 tx=0).stop 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 -- 172.21.15.19:0/3426659382 shutdown_connections 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55652f0 0x55f2e5567aa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55649d0 0x55f2e5567560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 --2- 172.21.15.19:0/3426659382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e5561340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 -- 172.21.15.19:0/3426659382 >> 172.21.15.19:0/3426659382 conn(0x55f2e556f7d0 msgr2=0x55f2e556b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 -- 172.21.15.19:0/3426659382 shutdown_connections 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.686+0000 7facc5539740 1 -- 172.21.15.19:0/3426659382 wait complete. 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 1 -- start start 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 1 Processor -- start 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e558a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55649d0 0x55f2e5583830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55652f0 0x55f2e5583d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2e5581ab0 con 0x55f2e55649d0 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f2e55813e0 con 0x55f2e55652f0 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2e5582180 con 0x55f2e5560f60 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facc5539740 10 client.0 _ll_register_callbacks cb 0x7faca4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55649d0 0x55f2e5583830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.687+0000 7facbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55649d0 0x55f2e5583830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46510/0 (socket says 172.21.15.19:46510) 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facc0efc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e558a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facbbfff640 1 -- 172.21.15.19:0/2886005140 learned_addr learned my addr 172.21.15.19:0/2886005140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facc16fd640 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55652f0 0x55f2e5583d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facbbfff640 1 -- 172.21.15.19:0/2886005140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 msgr2=0x55f2e558a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facbbfff640 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e558a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facbbfff640 1 -- 172.21.15.19:0/2886005140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55652f0 msgr2=0x55f2e5583d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facbbfff640 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55652f0 0x55f2e5583d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facbbfff640 1 -- 172.21.15.19:0/2886005140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2e5584400 con 0x55f2e55649d0 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facc0efc640 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e558a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facbbfff640 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55649d0 0x55f2e5583830 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faca4001eb0 tx=0x7faca401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a6f8991f7a7d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7fac9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7fac9bfff640 1 -- 172.21.15.19:0/2886005140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca400fe60 con 0x55f2e55649d0 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7fac9bfff640 1 -- 172.21.15.19:0/2886005140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7faca401c630 con 0x55f2e55649d0 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facc5539740 1 -- 172.21.15.19:0/2886005140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb4036050 con 0x55f2e55649d0 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7fac9bfff640 1 -- 172.21.15.19:0/2886005140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faca40050b0 con 0x55f2e55649d0 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facc5539740 10 client.15207 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.688+0000 7facc5539740 1 -- 172.21.15.19:0/2886005140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2e55eaed0 con 0x55f2e55649d0 2024-09-19T04:06:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7fac9bfff640 1 -- 172.21.15.19:0/2886005140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb4036050 con 0x55f2e55649d0 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7fac9bfff640 1 -- 172.21.15.19:0/2886005140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7faca4003c00 con 0x55f2e55649d0 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7facc5539740 10 client.15207 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7facc5539740 10 client.15207 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7facc5539740 20 client.15207 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7facc5539740 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55f2e556fbb0 0x55f2e55ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7facc5539740 4 client.15207 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7facc5539740 1 -- 172.21.15.19:0/2886005140 --> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] -- command(tid 0: {"prefix": "status"}) -- 0x55f2e54d2b10 con 0x55f2e556fbb0 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7facc0efc640 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55f2e556fbb0 0x55f2e55ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7facc0efc640 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55f2e556fbb0 0x55f2e55ee680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23c33caebe7d7dec server_cookie=b41770d70d876911 in_seq=0 out_seq=0 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.689+0000 7fac9bfff640 10 client.15207 ms_handle_connect on v2:172.21.15.111:6836/1580295791 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7fac9bfff640 1 -- 172.21.15.19:0/2886005140 <== mds.3 v2:172.21.15.111:6836/1580295791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f2e54d2b10 con 0x55f2e556fbb0 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7fac9bfff640 10 client.15207 handle_command_reply: tid=0 2024-09-19T04:06:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 client.15207 shutdown 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 -- 172.21.15.19:0/2886005140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55649d0 msgr2=0x55f2e5583830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55649d0 0x55f2e5583830 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faca4001eb0 tx=0x7faca401cdb0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 -- 172.21.15.19:0/2886005140 shutdown_connections 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x55f2e556fbb0 0x55f2e55ee680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2e55652f0 0x55f2e5583d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2e55649d0 0x55f2e5583830 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7fac9bfff640 0 client.15207 ms_handle_reset on v2:172.21.15.111:6836/1580295791 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 --2- 172.21.15.19:0/2886005140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2e5560f60 0x55f2e558a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 -- 172.21.15.19:0/2886005140 >> 172.21.15.19:0/2886005140 conn(0x55f2e556f7d0 msgr2=0x55f2e5566470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 -- 172.21.15.19:0/2886005140 shutdown_connections 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 1 -- 172.21.15.19:0/2886005140 wait complete. 2024-09-19T04:06:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.690+0000 7facc5539740 20 client.15207 trim_cache size 0 max 16384 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24452, 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 93.186224257000006, 2024-09-19T04:06:54.705 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:54.706 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:06:54.706 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:54.706 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 155.92042268099999 2024-09-19T04:06:54.706 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 -- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7faca80a3240 msgr2=0x7faca80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7faca80a3240 0x7faca80a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facb4006010 tx=0x7facb4005c50 comp rx=0 tx=0).stop 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 -- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc101de0 msgr2=0x7facbc1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc101de0 0x7facbc1984d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7facac00c890 tx=0x7facac00cd50 comp rx=0 tx=0).stop 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 -- 172.21.15.19:0/3674960853 shutdown_connections 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7faca80a3240 0x7faca80a5700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facbc106f10 0x7facbc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facbc102790 0x7facbc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 --2- 172.21.15.19:0/3674960853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facbc101de0 0x7facbc1984d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 -- 172.21.15.19:0/3674960853 >> 172.21.15.19:0/3674960853 conn(0x7facbc0fd8c0 msgr2=0x7facbc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 -- 172.21.15.19:0/3674960853 shutdown_connections 2024-09-19T04:06:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:54.691+0000 7facc3160640 1 -- 172.21.15.19:0/3674960853 wait complete. 2024-09-19T04:06:55.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:06:55.615 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:56.041 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:55 smithi019 ceph-mon[30938]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 88 B/s wr, 9 op/s 2024-09-19T04:06:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 -- 172.21.15.19:0/1017616235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 msgr2=0x7f4448105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 --2- 172.21.15.19:0/1017616235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 0x7f4448105920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f443c009900 tx=0x7f443c02a470 comp rx=0 tx=0).stop 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 -- 172.21.15.19:0/1017616235 shutdown_connections 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 --2- 172.21.15.19:0/1017616235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4448105f40 0x7f44481063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 --2- 172.21.15.19:0/1017616235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 0x7f4448105920 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 --2- 172.21.15.19:0/1017616235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4448100b30 0x7f4448100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 -- 172.21.15.19:0/1017616235 >> 172.21.15.19:0/1017616235 conn(0x7f44480fc7a0 msgr2=0x7f44480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 -- 172.21.15.19:0/1017616235 shutdown_connections 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.122+0000 7f444fe9a640 1 -- 172.21.15.19:0/1017616235 wait complete. 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444fe9a640 1 Processor -- start 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444fe9a640 1 -- start start 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444fe9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4448100b30 0x7f44481a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444fe9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 0x7f44481a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444fe9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4448105f40 0x7f444819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444fe9a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44481185c0 con 0x7f4448101450 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444fe9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4448118440 con 0x7f4448105f40 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444fe9a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4448118740 con 0x7f4448100b30 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444d435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 0x7f44481a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444d435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 0x7f44481a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46530/0 (socket says 172.21.15.19:46530) 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444d435640 1 -- 172.21.15.19:0/3306259688 learned_addr learned my addr 172.21.15.19:0/3306259688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444e437640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4448105f40 0x7f444819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.123+0000 7f444dc36640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4448100b30 0x7f44481a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444d435640 1 -- 172.21.15.19:0/3306259688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4448100b30 msgr2=0x7f44481a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444d435640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4448100b30 0x7f44481a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444d435640 1 -- 172.21.15.19:0/3306259688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4448105f40 msgr2=0x7f444819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444d435640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4448105f40 0x7f444819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444d435640 1 -- 172.21.15.19:0/3306259688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444819f020 con 0x7f4448101450 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444e437640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4448105f40 0x7f444819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444d435640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 0x7f44481a4c70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f443c005ab0 tx=0x7f443c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f4446ffd640 1 -- 172.21.15.19:0/3306259688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c03f030 con 0x7f4448101450 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f4446ffd640 1 -- 172.21.15.19:0/3306259688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f443c003ce0 con 0x7f4448101450 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444fe9a640 1 -- 172.21.15.19:0/3306259688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444819f2b0 con 0x7f4448101450 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f4446ffd640 1 -- 172.21.15.19:0/3306259688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c035070 con 0x7f4448101450 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.124+0000 7f444fe9a640 1 -- 172.21.15.19:0/3306259688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44481ab600 con 0x7f4448101450 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.126+0000 7f4446ffd640 1 -- 172.21.15.19:0/3306259688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f443c042030 con 0x7f4448101450 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4452273740 1 Processor -- start 2024-09-19T04:06:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4452273740 1 -- start start 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4452273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6519d0 0x55864c651db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4452273740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6522f0 0x55864c65c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4452273740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c65cbb0 0x55864c65efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4452273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55864c5b8090 con 0x55864c6519d0 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4452273740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55864c55fa70 con 0x55864c65cbb0 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4452273740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55864c5b4050 con 0x55864c6522f0 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f444e437640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6519d0 0x55864c651db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f444e437640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6519d0 0x55864c651db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46538/0 (socket says 172.21.15.19:46538) 2024-09-19T04:06:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f444e437640 1 -- 172.21.15.19:0/307152973 learned_addr learned my addr 172.21.15.19:0/307152973 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.127+0000 7f4446ffd640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f44300a3240 0x7f44300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444dc36640 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6522f0 0x55864c65c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444e437640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f44300a3240 0x7f44300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f4446ffd640 1 -- 172.21.15.19:0/3306259688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f443c0f0510 con 0x7f4448101450 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444d435640 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c65cbb0 0x55864c65efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444e437640 1 -- 172.21.15.19:0/307152973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6522f0 msgr2=0x55864c65c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444e437640 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6522f0 0x55864c65c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444e437640 1 -- 172.21.15.19:0/307152973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c65cbb0 msgr2=0x55864c65efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444e437640 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c65cbb0 0x55864c65efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444e437640 1 -- 172.21.15.19:0/307152973 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864c56c130 con 0x55864c6519d0 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444dc36640 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6522f0 0x55864c65c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444d435640 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c65cbb0 0x55864c65efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f444e437640 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6519d0 0x55864c651db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44400143c0 tx=0x7f4440014880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d55c5d89f23a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f442ffff640 1 -- 172.21.15.19:0/307152973 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444001b030 con 0x55864c6519d0 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f442ffff640 1 -- 172.21.15.19:0/307152973 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4440010040 con 0x55864c6519d0 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.128+0000 7f442ffff640 1 -- 172.21.15.19:0/307152973 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444000f050 con 0x55864c6519d0 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 -- 172.21.15.19:0/307152973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6519d0 msgr2=0x55864c651db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6519d0 0x55864c651db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44400143c0 tx=0x7f4440014880 comp rx=0 tx=0).stop 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f444e437640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f44300a3240 0x7f44300a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f444001e000 tx=0x7f444001a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 -- 172.21.15.19:0/307152973 shutdown_connections 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c65cbb0 0x55864c65efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6522f0 0x55864c65c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 --2- 172.21.15.19:0/307152973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6519d0 0x55864c651db0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 -- 172.21.15.19:0/307152973 >> 172.21.15.19:0/307152973 conn(0x55864c64df60 msgr2=0x55864c64e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 -- 172.21.15.19:0/307152973 shutdown_connections 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 -- 172.21.15.19:0/307152973 wait complete. 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 -- start start 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.129+0000 7f4452273740 1 Processor -- start 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f4452273740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c65c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f4452273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c6597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f4452273740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 0x55864c659d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f4452273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55864c65f780 con 0x55864c6522f0 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f4452273740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55864c572f20 con 0x55864c6519d0 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f4452273740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55864c65f900 con 0x55864c65cbb0 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f4452273740 10 client.0 _ll_register_callbacks cb 0x55864c53c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444d435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c6597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444d435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c6597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46546/0 (socket says 172.21.15.19:46546) 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444d435640 1 -- 172.21.15.19:0/3622544651 learned_addr learned my addr 172.21.15.19:0/3622544651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444dc36640 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c65c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444e437640 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 0x55864c659d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444d435640 1 -- 172.21.15.19:0/3622544651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 msgr2=0x55864c659d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444d435640 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 0x55864c659d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444d435640 1 -- 172.21.15.19:0/3622544651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 msgr2=0x55864c65c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444d435640 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c65c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444d435640 1 -- 172.21.15.19:0/3622544651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864c3292b0 con 0x55864c6522f0 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.130+0000 7f444dc36640 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c65c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f444d435640 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c6597d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f443c000c00 tx=0x7f443c0bc620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fd3325845fbd1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f442d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f442d7fa640 1 -- 172.21.15.19:0/3622544651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c049040 con 0x55864c6522f0 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 1 -- 172.21.15.19:0/3622544651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4440010040 con 0x55864c6522f0 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f442d7fa640 1 -- 172.21.15.19:0/3622544651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f443c03b460 con 0x55864c6522f0 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f442d7fa640 1 -- 172.21.15.19:0/3622544651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f443c0bce90 con 0x55864c6522f0 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 10 client.15219 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 1 -- 172.21.15.19:0/3622544651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55864c66b040 con 0x55864c6522f0 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f442d7fa640 1 -- 172.21.15.19:0/3622544651 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4440010040 con 0x55864c6522f0 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f442d7fa640 1 -- 172.21.15.19:0/3622544651 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f443c0b8e00 con 0x55864c6522f0 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 10 client.15219 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 10 client.15219 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 20 client.15219 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55864c66d5c0 0x55864c66ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 4 client.15219 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.131+0000 7f4452273740 1 -- 172.21.15.19:0/3622544651 --> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55864c5c5600 con 0x55864c66d5c0 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.132+0000 7f444dc36640 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55864c66d5c0 0x55864c66ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.132+0000 7f444dc36640 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55864c66d5c0 0x55864c66ea30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cf158b0de269581 server_cookie=dc4db3a771681762 in_seq=0 out_seq=0 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.132+0000 7f442d7fa640 10 client.15219 ms_handle_connect on v2:172.21.15.196:6836/1041250110 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f442d7fa640 1 -- 172.21.15.19:0/3622544651 <== mds.2 v2:172.21.15.196:6836/1041250110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55864c5c5600 con 0x55864c66d5c0 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f442d7fa640 10 client.15219 handle_command_reply: tid=0 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 client.15219 shutdown 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 -- 172.21.15.19:0/3622544651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 msgr2=0x55864c6597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c6597d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f443c000c00 tx=0x7f443c0bc620 comp rx=0 tx=0).stop 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 -- 172.21.15.19:0/3622544651 shutdown_connections 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55864c66d5c0 0x55864c66ea30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 0x55864c659d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f442d7fa640 0 client.15219 ms_handle_reset on v2:172.21.15.196:6836/1041250110 2024-09-19T04:06:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c6597d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 --2- 172.21.15.19:0/3622544651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c65c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 -- 172.21.15.19:0/3622544651 >> 172.21.15.19:0/3622544651 conn(0x55864c64df60 msgr2=0x55864c653c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 -- 172.21.15.19:0/3622544651 shutdown_connections 2024-09-19T04:06:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 1 -- 172.21.15.19:0/3622544651 wait complete. 2024-09-19T04:06:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.133+0000 7f4452273740 20 client.15219 trim_cache size 0 max 16384 2024-09-19T04:06:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.151+0000 7f4452273740 1 Processor -- start 2024-09-19T04:06:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.151+0000 7f4452273740 1 -- start start 2024-09-19T04:06:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.151+0000 7f4452273740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c66ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.151+0000 7f4452273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c648ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.151+0000 7f4452273740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 0x55864c6580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.151+0000 7f4452273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55864c66cd70 con 0x55864c6522f0 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.151+0000 7f4452273740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55864c66d440 con 0x55864c6519d0 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.151+0000 7f4452273740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55864c66db10 con 0x55864c65cbb0 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c648ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c648ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46554/0 (socket says 172.21.15.19:46554) 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 -- 172.21.15.19:0/1923097254 learned_addr learned my addr 172.21.15.19:0/1923097254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444e437640 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 0x55864c6580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444dc36640 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c66ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 -- 172.21.15.19:0/1923097254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 msgr2=0x55864c6580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 0x55864c6580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 -- 172.21.15.19:0/1923097254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 msgr2=0x55864c66ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c66ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 -- 172.21.15.19:0/1923097254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864c651800 con 0x55864c6522f0 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444dc36640 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c66ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f444d435640 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c648ce0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f443c0c4ed0 tx=0x7f443c0baca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f533ab28519a710 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f442f7fe640 1 -- 172.21.15.19:0/1923097254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c049040 con 0x55864c6522f0 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f442f7fe640 1 -- 172.21.15.19:0/1923097254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f443c0bc790 con 0x55864c6522f0 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.152+0000 7f442f7fe640 1 -- 172.21.15.19:0/1923097254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0f0240 con 0x55864c6522f0 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 -- 172.21.15.19:0/1923097254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 msgr2=0x55864c648ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c648ce0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f443c0c4ed0 tx=0x7f443c0baca0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 -- 172.21.15.19:0/1923097254 shutdown_connections 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c65cbb0 0x55864c6580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c6522f0 0x55864c648ce0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 --2- 172.21.15.19:0/1923097254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6519d0 0x55864c66ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 -- 172.21.15.19:0/1923097254 >> 172.21.15.19:0/1923097254 conn(0x55864c64df60 msgr2=0x55864c659790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 -- 172.21.15.19:0/1923097254 shutdown_connections 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 -- 172.21.15.19:0/1923097254 wait complete. 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 -- start start 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.153+0000 7f4452273740 1 Processor -- start 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f4452273740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6519d0 0x55864c6771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f4452273740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6522f0 0x55864c656350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f4452273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c65cbb0 0x55864c656890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f4452273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55864c66e8b0 con 0x55864c65cbb0 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f4452273740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55864c66e1e0 con 0x55864c6522f0 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f4452273740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55864c66ef80 con 0x55864c6519d0 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f4452273740 10 client.0 _ll_register_callbacks cb 0x7f443c032b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444e437640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c65cbb0 0x55864c656890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444e437640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c65cbb0 0x55864c656890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46566/0 (socket says 172.21.15.19:46566) 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444e437640 1 -- 172.21.15.19:0/1476226901 learned_addr learned my addr 172.21.15.19:0/1476226901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444d435640 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6522f0 0x55864c656350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444dc36640 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6519d0 0x55864c6771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444e437640 1 -- 172.21.15.19:0/1476226901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6519d0 msgr2=0x55864c6771e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444e437640 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6519d0 0x55864c6771e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444e437640 1 -- 172.21.15.19:0/1476226901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6522f0 msgr2=0x55864c656350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444e437640 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6522f0 0x55864c656350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444e437640 1 -- 172.21.15.19:0/1476226901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864c656e50 con 0x55864c65cbb0 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.154+0000 7f444d435640 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6522f0 0x55864c656350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f444e437640 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c65cbb0 0x55864c656890 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55864c64ed00 tx=0x7f4440006ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c11260b836d9253 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f442d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f442d7fa640 1 -- 172.21.15.19:0/1476226901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444002c070 con 0x55864c65cbb0 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f442d7fa640 1 -- 172.21.15.19:0/1476226901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f44400177f0 con 0x55864c65cbb0 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 1 -- 172.21.15.19:0/1476226901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443c0bc790 con 0x55864c65cbb0 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f442d7fa640 1 -- 172.21.15.19:0/1476226901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f444001b040 con 0x55864c65cbb0 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 10 client.15225 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 1 -- 172.21.15.19:0/1476226901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55864c675b20 con 0x55864c65cbb0 2024-09-19T04:06:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f442d7fa640 1 -- 172.21.15.19:0/1476226901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f443c0bc790 con 0x55864c65cbb0 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f442d7fa640 1 -- 172.21.15.19:0/1476226901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4440020ab0 con 0x55864c65cbb0 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 10 client.15225 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 10 client.15225 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 20 client.15225 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55864c6dc580 0x55864c6db290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 4 client.15225 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.155+0000 7f4452273740 1 -- 172.21.15.19:0/1476226901 --> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] -- command(tid 0: {"prefix": "status"}) -- 0x55864c5bfb10 con 0x55864c6dc580 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.156+0000 7f444dc36640 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55864c6dc580 0x55864c6db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.156+0000 7f444dc36640 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55864c6dc580 0x55864c6db290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fcec7b151f5b6a4a server_cookie=795b1a8c32ed56d7 in_seq=0 out_seq=0 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.156+0000 7f442d7fa640 10 client.15225 ms_handle_connect on v2:172.21.15.196:6836/1041250110 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.156+0000 7f442d7fa640 1 -- 172.21.15.19:0/1476226901 <== mds.2 v2:172.21.15.196:6836/1041250110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55864c5bfb10 con 0x55864c6dc580 2024-09-19T04:06:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.156+0000 7f442d7fa640 10 client.15225 handle_command_reply: tid=0 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.156+0000 7f4452273740 1 client.15225 shutdown 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.156+0000 7f4452273740 1 -- 172.21.15.19:0/1476226901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c65cbb0 msgr2=0x55864c656890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.156+0000 7f4452273740 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c65cbb0 0x55864c656890 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55864c64ed00 tx=0x7f4440006ec0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 1 -- 172.21.15.19:0/1476226901 shutdown_connections 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x55864c6dc580 0x55864c6db290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55864c65cbb0 0x55864c656890 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55864c6522f0 0x55864c656350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 1 --2- 172.21.15.19:0/1476226901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864c6519d0 0x55864c6771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f442d7fa640 0 client.15225 ms_handle_reset on v2:172.21.15.196:6836/1041250110 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 1 -- 172.21.15.19:0/1476226901 >> 172.21.15.19:0/1476226901 conn(0x55864c64df60 msgr2=0x55864c653420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 1 -- 172.21.15.19:0/1476226901 shutdown_connections 2024-09-19T04:06:56.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 1 -- 172.21.15.19:0/1476226901 wait complete. 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f4452273740 20 client.15225 trim_cache size 0 max 16384 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24454, 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 95.658441152999998, 2024-09-19T04:06:56.174 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:56.174 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:06:56.174 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:56.174 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 160.19824371999999 2024-09-19T04:06:56.174 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 -- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f44300a3240 msgr2=0x7f44300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f44300a3240 0x7f44300a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f444001e000 tx=0x7f444001a000 comp rx=0 tx=0).stop 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 -- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 msgr2=0x7f44481a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 0x7f44481a4c70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f443c005ab0 tx=0x7f443c0042f0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 -- 172.21.15.19:0/3306259688 shutdown_connections 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f44300a3240 0x7f44300a5700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4448105f40 0x7f444819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4448101450 0x7f44481a4c70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 --2- 172.21.15.19:0/3306259688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4448100b30 0x7f44481a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:56.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.157+0000 7f444fe9a640 1 -- 172.21.15.19:0/3306259688 >> 172.21.15.19:0/3306259688 conn(0x7f44480fc7a0 msgr2=0x7f44481038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:56.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.158+0000 7f444fe9a640 1 -- 172.21.15.19:0/3306259688 shutdown_connections 2024-09-19T04:06:56.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:56.158+0000 7f444fe9a640 1 -- 172.21.15.19:0/3306259688 wait complete. 2024-09-19T04:06:56.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:55 smithi196 ceph-mon[35731]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 88 B/s wr, 9 op/s 2024-09-19T04:06:56.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:55 smithi111 ceph-mon[35365]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 88 B/s wr, 9 op/s 2024-09-19T04:06:56.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:06:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 -- 172.21.15.19:0/1159245486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 msgr2=0x7f6bdc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/1159245486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 0x7f6bdc1067e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009900 tx=0x7f6bd002a410 comp rx=0 tx=0).stop 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 -- 172.21.15.19:0/1159245486 shutdown_connections 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/1159245486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bdc106f10 0x7f6bdc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/1159245486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 0x7f6bdc1067e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/1159245486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bdc101de0 0x7f6bdc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 -- 172.21.15.19:0/1159245486 >> 172.21.15.19:0/1159245486 conn(0x7f6bdc0fd8c0 msgr2=0x7f6bdc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 -- 172.21.15.19:0/1159245486 shutdown_connections 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 -- 172.21.15.19:0/1159245486 wait complete. 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 Processor -- start 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.597+0000 7f6be3e9a640 1 -- start start 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be3e9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bdc101de0 0x7f6bdc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be3e9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 0x7f6bdc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be3e9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bdc106f10 0x7f6bdc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be3e9a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc111f90 con 0x7f6bdc102790 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be3e9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc111e10 con 0x7f6bdc101de0 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be3e9a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc112110 con 0x7f6bdc106f10 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 0x7f6bdc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 0x7f6bdc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46586/0 (socket says 172.21.15.19:46586) 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 -- 172.21.15.19:0/2132908711 learned_addr learned my addr 172.21.15.19:0/2132908711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be2437640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bdc106f10 0x7f6bdc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1c36640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bdc101de0 0x7f6bdc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 -- 172.21.15.19:0/2132908711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bdc106f10 msgr2=0x7f6bdc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bdc106f10 0x7f6bdc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 -- 172.21.15.19:0/2132908711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bdc101de0 msgr2=0x7f6bdc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bdc101de0 0x7f6bdc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 -- 172.21.15.19:0/2132908711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc1a6990 con 0x7f6bdc102790 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be2437640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bdc106f10 0x7f6bdc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6be1435640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 0x7f6bdc074a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0006010 tx=0x7f6bd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.598+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2132908711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd003f030 con 0x7f6bdc102790 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.599+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2132908711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd0003ce0 con 0x7f6bdc102790 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.599+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2132908711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0035070 con 0x7f6bdc102790 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.599+0000 7f6be3e9a640 1 -- 172.21.15.19:0/2132908711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc1a6c20 con 0x7f6bdc102790 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.599+0000 7f6be3e9a640 1 -- 172.21.15.19:0/2132908711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a7100 con 0x7f6bdc102790 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2132908711 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6bd0042030 con 0x7f6bdc102790 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be6273740 1 Processor -- start 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be6273740 1 -- start start 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be6273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be6273740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be6273740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be6273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559351658090 con 0x5593516f19d0 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be6273740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5593515ffa70 con 0x5593516f22f0 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be6273740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559351654050 con 0x5593516fcbb0 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6bdaffd640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6bc40a3240 0x7f6bc40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46596/0 (socket says 172.21.15.19:46596) 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 -- 172.21.15.19:0/4212804060 learned_addr learned my addr 172.21.15.19:0/4212804060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6bc40a3240 0x7f6bc40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be1435640 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be1c36640 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2132908711 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6bd00f0270 con 0x7f6bdc102790 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 -- 172.21.15.19:0/4212804060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 msgr2=0x5593516fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516fefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 -- 172.21.15.19:0/4212804060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 msgr2=0x5593516fc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516fc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 -- 172.21.15.19:0/4212804060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935160c130 con 0x5593516f19d0 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be1c36640 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516f1db0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00afa0 tx=0x7f6bcc016700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634975a84a72a8a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.602+0000 7f6be2437640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6bc40a3240 0x7f6bc40a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00afd0 tx=0x7f6bcc01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6bc3fff640 1 -- 172.21.15.19:0/4212804060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc026070 con 0x5593516f19d0 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6bc3fff640 1 -- 172.21.15.19:0/4212804060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc009d00 con 0x5593516f19d0 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6bc3fff640 1 -- 172.21.15.19:0/4212804060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc013040 con 0x5593516f19d0 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 -- 172.21.15.19:0/4212804060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 msgr2=0x5593516f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516f1db0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00afa0 tx=0x7f6bcc016700 comp rx=0 tx=0).stop 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 -- 172.21.15.19:0/4212804060 shutdown_connections 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 --2- 172.21.15.19:0/4212804060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516f1db0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 -- 172.21.15.19:0/4212804060 >> 172.21.15.19:0/4212804060 conn(0x5593516edf60 msgr2=0x5593516ee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 -- 172.21.15.19:0/4212804060 shutdown_connections 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.603+0000 7f6be6273740 1 -- 172.21.15.19:0/4212804060 wait complete. 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 1 -- start start 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 1 Processor -- start 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5593516ff780 con 0x5593516f19d0 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559351612f20 con 0x5593516f22f0 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5593516ff900 con 0x5593516fcbb0 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be6273740 10 client.0 _ll_register_callbacks cb 0x5593515dc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.604+0000 7f6be1c36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1c36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516fc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46598/0 (socket says 172.21.15.19:46598) 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1c36640 1 -- 172.21.15.19:0/2219596646 learned_addr learned my addr 172.21.15.19:0/2219596646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1435640 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1c36640 1 -- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 msgr2=0x5593516f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be2437640 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1c36640 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1c36640 1 -- 172.21.15.19:0/2219596646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 msgr2=0x5593516f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1c36640 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1c36640 1 -- 172.21.15.19:0/2219596646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593513c92b0 con 0x5593516f19d0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be2437640 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be1c36640 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516fc430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400be50 tx=0x7f6bd40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c12344b6e07255d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6bc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6bc17fa640 1 -- 172.21.15.19:0/2219596646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4012650 con 0x5593516f19d0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6bc17fa640 1 -- 172.21.15.19:0/2219596646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6bd40127f0 con 0x5593516f19d0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be6273740 1 -- 172.21.15.19:0/2219596646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bcc009d00 con 0x5593516f19d0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6bc17fa640 1 -- 172.21.15.19:0/2219596646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bd40129f0 con 0x5593516f19d0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be6273740 10 client.15237 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.605+0000 7f6be6273740 1 -- 172.21.15.19:0/2219596646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55935170b0f0 con 0x5593516f19d0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6bc17fa640 1 -- 172.21.15.19:0/2219596646 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc009d00 con 0x5593516f19d0 2024-09-19T04:06:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6bc17fa640 1 -- 172.21.15.19:0/2219596646 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6bd401a3d0 con 0x5593516f19d0 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6be6273740 10 client.15237 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6be6273740 10 client.15237 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6be6273740 20 client.15237 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6be6273740 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55935170d7a0 0x55935170ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6be6273740 4 client.15237 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6be6273740 1 -- 172.21.15.19:0/2219596646 --> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559351665600 con 0x55935170d7a0 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.606+0000 7f6be1435640 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55935170d7a0 0x55935170ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.607+0000 7f6be1435640 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55935170d7a0 0x55935170ec10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=758e921a6ffb8bfd server_cookie=e343cf7ae4c96429 in_seq=0 out_seq=0 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.607+0000 7f6bc17fa640 10 client.15237 ms_handle_connect on v2:172.21.15.196:6838/1173261295 2024-09-19T04:06:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.607+0000 7f6bc17fa640 1 -- 172.21.15.19:0/2219596646 <== mds.1 v2:172.21.15.196:6838/1173261295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559351665600 con 0x55935170d7a0 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.607+0000 7f6bc17fa640 10 client.15237 handle_command_reply: tid=0 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.607+0000 7f6be6273740 1 client.15237 shutdown 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 -- 172.21.15.19:0/2219596646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 msgr2=0x5593516fc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516fc430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400be50 tx=0x7f6bd40117b0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 -- 172.21.15.19:0/2219596646 shutdown_connections 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55935170d7a0 0x55935170ec10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x5593516f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f22f0 0x5593516f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6bc17fa640 0 client.15237 ms_handle_reset on v2:172.21.15.196:6838/1173261295 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 --2- 172.21.15.19:0/2219596646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593516fc430 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 -- 172.21.15.19:0/2219596646 >> 172.21.15.19:0/2219596646 conn(0x5593516edf60 msgr2=0x5593516f3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 -- 172.21.15.19:0/2219596646 shutdown_connections 2024-09-19T04:06:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 1 -- 172.21.15.19:0/2219596646 wait complete. 2024-09-19T04:06:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.608+0000 7f6be6273740 20 client.15237 trim_cache size 0 max 16384 2024-09-19T04:06:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be6273740 1 Processor -- start 2024-09-19T04:06:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be6273740 1 -- start start 2024-09-19T04:06:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be6273740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f19d0 0x5593516f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be6273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f22f0 0x5593516f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be6273740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x559351716ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be6273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55935170cf50 con 0x5593516f22f0 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be6273740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55935170d620 con 0x5593516f19d0 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be6273740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55935170dcf0 con 0x5593516fcbb0 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be1435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f22f0 0x5593516f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be1435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f22f0 0x5593516f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46610/0 (socket says 172.21.15.19:46610) 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be1435640 1 -- 172.21.15.19:0/2721351325 learned_addr learned my addr 172.21.15.19:0/2721351325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be1c36640 1 --2- 172.21.15.19:0/2721351325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f19d0 0x5593516f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be1435640 1 -- 172.21.15.19:0/2721351325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 msgr2=0x559351716ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be1435640 1 --2- 172.21.15.19:0/2721351325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x559351716ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be1435640 1 -- 172.21.15.19:0/2721351325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f19d0 msgr2=0x5593516f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.626+0000 7f6be1435640 1 --2- 172.21.15.19:0/2721351325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f19d0 0x5593516f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.627+0000 7f6be1435640 1 -- 172.21.15.19:0/2721351325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593516f1800 con 0x5593516f22f0 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.627+0000 7f6be1435640 1 --2- 172.21.15.19:0/2721351325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f22f0 0x5593516f6f20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00eda20 tx=0x7f6bd0040fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6857222dbfaedcec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.627+0000 7f6bc37fe640 1 -- 172.21.15.19:0/2721351325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0049040 con 0x5593516f22f0 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.627+0000 7f6bc37fe640 1 -- 172.21.15.19:0/2721351325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd003b580 con 0x5593516f22f0 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.627+0000 7f6bc37fe640 1 -- 172.21.15.19:0/2721351325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd00c0960 con 0x5593516f22f0 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.627+0000 7f6be6273740 1 -- 172.21.15.19:0/2721351325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f22f0 msgr2=0x5593516f6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 --2- 172.21.15.19:0/2721351325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f22f0 0x5593516f6f20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00eda20 tx=0x7f6bd0040fb0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 -- 172.21.15.19:0/2721351325 shutdown_connections 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 --2- 172.21.15.19:0/2721351325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516fcbb0 0x559351716ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 --2- 172.21.15.19:0/2721351325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f22f0 0x5593516f6f20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 --2- 172.21.15.19:0/2721351325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516f19d0 0x5593516f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 -- 172.21.15.19:0/2721351325 >> 172.21.15.19:0/2721351325 conn(0x5593516edf60 msgr2=0x5593516f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 -- 172.21.15.19:0/2721351325 shutdown_connections 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 -- 172.21.15.19:0/2721351325 wait complete. 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 -- start start 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 Processor -- start 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593517123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516f22f0 0x5593517128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516fcbb0 0x559351775ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.628+0000 7f6be6273740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55935170ea90 con 0x5593516f19d0 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be6273740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55935170e3c0 con 0x5593516fcbb0 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be6273740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55935170f160 con 0x5593516f22f0 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be6273740 10 client.0 _ll_register_callbacks cb 0x7f6bd4005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593517123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593517123a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46622/0 (socket says 172.21.15.19:46622) 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 -- 172.21.15.19:0/3972618339 learned_addr learned my addr 172.21.15.19:0/3972618339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be2437640 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516fcbb0 0x559351775ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1435640 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516f22f0 0x5593517128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 -- 172.21.15.19:0/3972618339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516f22f0 msgr2=0x5593517128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516f22f0 0x5593517128e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 -- 172.21.15.19:0/3972618339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516fcbb0 msgr2=0x559351775ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516fcbb0 0x559351775ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 -- 172.21.15.19:0/3972618339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593517763e0 con 0x5593516f19d0 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be1c36640 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593517123a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400be50 tx=0x7f6bd401a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bcc532eb7bdbc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6bc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6bc17fa640 1 -- 172.21.15.19:0/3972618339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd401aab0 con 0x5593516f19d0 2024-09-19T04:06:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6bc17fa640 1 -- 172.21.15.19:0/3972618339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6bd401ac50 con 0x5593516f19d0 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.629+0000 7f6be6273740 1 -- 172.21.15.19:0/3972618339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55935165fb10 con 0x5593516f19d0 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6bc17fa640 1 -- 172.21.15.19:0/3972618339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bd403b050 con 0x5593516f19d0 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6be6273740 10 client.15243 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6be6273740 1 -- 172.21.15.19:0/3972618339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559351777720 con 0x5593516f19d0 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6bc17fa640 1 -- 172.21.15.19:0/3972618339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55935165fb10 con 0x5593516f19d0 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6bc17fa640 1 -- 172.21.15.19:0/3972618339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6bd40050d0 con 0x5593516f19d0 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6be6273740 10 client.15243 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6be6273740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6be6273740 20 client.15243 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6be6273740 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55935177c2a0 0x55935177afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6be6273740 4 client.15243 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.630+0000 7f6be6273740 1 -- 172.21.15.19:0/3972618339 --> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] -- command(tid 0: {"prefix": "status"}) -- 0x5593515dc0e0 con 0x55935177c2a0 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.631+0000 7f6be1435640 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55935177c2a0 0x55935177afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.631+0000 7f6be1435640 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55935177c2a0 0x55935177afb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b283b66f0073e48 server_cookie=b41ddcf2d8b4fd30 in_seq=0 out_seq=0 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.631+0000 7f6bc17fa640 10 client.15243 ms_handle_connect on v2:172.21.15.196:6838/1173261295 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.631+0000 7f6bc17fa640 1 -- 172.21.15.19:0/3972618339 <== mds.1 v2:172.21.15.196:6838/1173261295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5593515dc0e0 con 0x55935177c2a0 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.631+0000 7f6bc17fa640 10 client.15243 handle_command_reply: tid=0 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.631+0000 7f6be6273740 1 client.15243 shutdown 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 -- 172.21.15.19:0/3972618339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 msgr2=0x5593517123a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593517123a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400be50 tx=0x7f6bd401a3d0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 -- 172.21.15.19:0/3972618339 shutdown_connections 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x55935177c2a0 0x55935177afb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5593516fcbb0 0x559351775ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593516f22f0 0x5593517128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 --2- 172.21.15.19:0/3972618339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593516f19d0 0x5593517123a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6bc17fa640 0 client.15243 ms_handle_reset on v2:172.21.15.196:6838/1173261295 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 -- 172.21.15.19:0/3972618339 >> 172.21.15.19:0/3972618339 conn(0x5593516edf60 msgr2=0x5593516ec800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 -- 172.21.15.19:0/3972618339 shutdown_connections 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 1 -- 172.21.15.19:0/3972618339 wait complete. 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be6273740 20 client.15243 trim_cache size 0 max 16384 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-09-19T04:06:57.647 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24469, 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 98.142050237999996, 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 156.300240602 2024-09-19T04:06:57.648 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 -- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6bc40a3240 msgr2=0x7f6bc40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6bc40a3240 0x7f6bc40a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00afd0 tx=0x7f6bcc01a000 comp rx=0 tx=0).stop 2024-09-19T04:06:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 -- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 msgr2=0x7f6bdc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 0x7f6bdc074a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0006010 tx=0x7f6bd00042f0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 -- 172.21.15.19:0/2132908711 shutdown_connections 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6bc40a3240 0x7f6bc40a5700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bdc106f10 0x7f6bdc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc102790 0x7f6bdc074a30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 --2- 172.21.15.19:0/2132908711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bdc101de0 0x7f6bdc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.632+0000 7f6be3e9a640 1 -- 172.21.15.19:0/2132908711 >> 172.21.15.19:0/2132908711 conn(0x7f6bdc0fd8c0 msgr2=0x7f6bdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.633+0000 7f6be3e9a640 1 -- 172.21.15.19:0/2132908711 shutdown_connections 2024-09-19T04:06:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:57.633+0000 7f6be3e9a640 1 -- 172.21.15.19:0/2132908711 wait complete. 2024-09-19T04:06:58.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:57 smithi019 ceph-mon[30938]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:06:58.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:57 smithi196 ceph-mon[35731]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:06:58.332 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:06:58.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:57 smithi111 ceph-mon[35365]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T04:06:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:06:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 -- 172.21.15.19:0/3003683646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0101dd0 msgr2=0x7ffad01021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/3003683646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0101dd0 0x7ffad01021b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac402a410 comp rx=0 tx=0).stop 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 -- 172.21.15.19:0/3003683646 shutdown_connections 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/3003683646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad0106f00 0x7ffad010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/3003683646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0102780 0x7ffad01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/3003683646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0101dd0 0x7ffad01021b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 -- 172.21.15.19:0/3003683646 >> 172.21.15.19:0/3003683646 conn(0x7ffad00fd890 msgr2=0x7ffad00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 -- 172.21.15.19:0/3003683646 shutdown_connections 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.056+0000 7ffad6ea1640 1 -- 172.21.15.19:0/3003683646 wait complete. 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad6ea1640 1 Processor -- start 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad6ea1640 1 -- start start 2024-09-19T04:06:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad6ea1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad0101dd0 0x7ffad0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad6ea1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0102780 0x7ffad01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad6ea1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0106f00 0x7ffad019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad6ea1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0111f20 con 0x7ffad0106f00 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad6ea1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0111da0 con 0x7ffad0102780 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad6ea1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01120a0 con 0x7ffad0101dd0 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0106f00 0x7ffad019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0106f00 0x7ffad019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46626/0 (socket says 172.21.15.19:46626) 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 -- 172.21.15.19:0/1281127504 learned_addr learned my addr 172.21.15.19:0/1281127504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad0101dd0 0x7ffad0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffacffff640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0102780 0x7ffad01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 -- 172.21.15.19:0/1281127504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad0101dd0 msgr2=0x7ffad0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad0101dd0 0x7ffad0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 -- 172.21.15.19:0/1281127504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0102780 msgr2=0x7ffad01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0102780 0x7ffad01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 -- 172.21.15.19:0/1281127504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad019d430 con 0x7ffad0106f00 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad0101dd0 0x7ffad0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffacffff640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0102780 0x7ffad01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.057+0000 7ffad543e640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0106f00 0x7ffad019cd50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffac800eaa0 tx=0x7ffac800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.058+0000 7ffacdffb640 1 -- 172.21.15.19:0/1281127504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac800cdb0 con 0x7ffad0106f00 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.058+0000 7ffacdffb640 1 -- 172.21.15.19:0/1281127504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac8005020 con 0x7ffad0106f00 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.058+0000 7ffad6ea1640 1 -- 172.21.15.19:0/1281127504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad019d720 con 0x7ffad0106f00 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.058+0000 7ffacdffb640 1 -- 172.21.15.19:0/1281127504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80116b0 con 0x7ffad0106f00 2024-09-19T04:06:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.058+0000 7ffad6ea1640 1 -- 172.21.15.19:0/1281127504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad01a5000 con 0x7ffad0106f00 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.060+0000 7ffacdffb640 1 -- 172.21.15.19:0/1281127504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ffac8005560 con 0x7ffad0106f00 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.060+0000 7ffad927a740 1 Processor -- start 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.060+0000 7ffad927a740 1 -- start start 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffad927a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce90db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffad927a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce912f0 0x558e9ce9b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffad927a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ce9dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffad927a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558e9cdf7090 con 0x558e9ce9bbb0 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffad927a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558e9cd9ea70 con 0x558e9ce909d0 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffad927a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e9cdf3050 con 0x558e9ce912f0 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacdffb640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a3240 0x7ffac00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ce9dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ce9dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46630/0 (socket says 172.21.15.19:46630) 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 -- 172.21.15.19:0/2720089951 learned_addr learned my addr 172.21.15.19:0/2720089951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce90db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffad543e640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a3240 0x7ffac00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce912f0 0x558e9ce9b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacdffb640 1 -- 172.21.15.19:0/1281127504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7ffac80cff50 con 0x7ffad0106f00 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 -- 172.21.15.19:0/2720089951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce912f0 msgr2=0x558e9ce9b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce912f0 0x558e9ce9b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 -- 172.21.15.19:0/2720089951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 msgr2=0x558e9ce90db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce90db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 -- 172.21.15.19:0/2720089951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9cdab130 con 0x558e9ce9bbb0 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.061+0000 7ffacffff640 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ce9dfa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00cfc0 tx=0x7ffabc00c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3827d1e89c17b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.062+0000 7ffad543e640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a3240 0x7ffac00a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffac8098ba0 tx=0x7ffac80916b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.062+0000 7ffabaffd640 1 -- 172.21.15.19:0/2720089951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc01a070 con 0x558e9ce9bbb0 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.062+0000 7ffabaffd640 1 -- 172.21.15.19:0/2720089951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffabc0140d0 con 0x558e9ce9bbb0 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.062+0000 7ffabaffd640 1 -- 172.21.15.19:0/2720089951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0035f0 con 0x558e9ce9bbb0 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.062+0000 7ffad927a740 1 -- 172.21.15.19:0/2720089951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 msgr2=0x558e9ce9dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.062+0000 7ffad927a740 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ce9dfa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00cfc0 tx=0x7ffabc00c570 comp rx=0 tx=0).stop 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 -- 172.21.15.19:0/2720089951 shutdown_connections 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ce9dfa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce912f0 0x558e9ce9b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 --2- 172.21.15.19:0/2720089951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce90db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 -- 172.21.15.19:0/2720089951 >> 172.21.15.19:0/2720089951 conn(0x558e9ce8cf60 msgr2=0x558e9ce8d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 -- 172.21.15.19:0/2720089951 shutdown_connections 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 -- 172.21.15.19:0/2720089951 wait complete. 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 -- start start 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 Processor -- start 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce9b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce912f0 0x558e9ce987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 0x558e9ce98cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558e9ce9e780 con 0x558e9ce912f0 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558e9cdb1f20 con 0x558e9ce909d0 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.063+0000 7ffad927a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e9ce9e900 con 0x558e9ce9bbb0 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad927a740 10 client.0 _ll_register_callbacks cb 0x558e9cd7b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce912f0 0x558e9ce987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce912f0 0x558e9ce987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46634/0 (socket says 172.21.15.19:46634) 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 -- 172.21.15.19:0/1102021184 learned_addr learned my addr 172.21.15.19:0/1102021184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffacffff640 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce9b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffacffff640 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 0x558e9ce98cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 -- 172.21.15.19:0/1102021184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 msgr2=0x558e9ce98cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 0x558e9ce98cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 -- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 msgr2=0x558e9ce9b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce9b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 -- 172.21.15.19:0/1102021184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9cb682b0 con 0x558e9ce912f0 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffacffff640 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce9b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:06:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce912f0 0x558e9ce987b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac40318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c76667450a8a09e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffaaffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffaaffff640 1 -- 172.21.15.19:0/1102021184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4035040 con 0x558e9ce912f0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffad927a740 1 -- 172.21.15.19:0/1102021184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffabc0140d0 con 0x558e9ce912f0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.064+0000 7ffaaffff640 1 -- 172.21.15.19:0/1102021184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7ffac4004240 con 0x558e9ce912f0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffaaffff640 1 -- 172.21.15.19:0/1102021184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffac4004440 con 0x558e9ce912f0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffad927a740 10 client.15255 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffad927a740 1 -- 172.21.15.19:0/1102021184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e9ceaa0f0 con 0x558e9ce912f0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffaaffff640 1 -- 172.21.15.19:0/1102021184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffabc0140d0 con 0x558e9ce912f0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffaaffff640 1 -- 172.21.15.19:0/1102021184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffac4004b60 con 0x558e9ce912f0 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffad927a740 10 client.15255 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffad927a740 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffad927a740 20 client.15255 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffad927a740 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x558e9ceac6f0 0x558e9ceadb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffad927a740 4 client.15255 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffad927a740 1 -- 172.21.15.19:0/1102021184 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e9ce04600 con 0x558e9ceac6f0 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.065+0000 7ffacffff640 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x558e9ceac6f0 0x558e9ceadb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.066+0000 7ffacffff640 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x558e9ceac6f0 0x558e9ceadb60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e53f7179bd1b0c server_cookie=9a4c423a1ff17e3a in_seq=0 out_seq=0 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.066+0000 7ffaaffff640 10 client.15255 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffaaffff640 1 -- 172.21.15.19:0/1102021184 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e9ce04600 con 0x558e9ceac6f0 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffaaffff640 10 client.15255 handle_command_reply: tid=0 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 client.15255 shutdown 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 -- 172.21.15.19:0/1102021184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce912f0 msgr2=0x558e9ce987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce912f0 0x558e9ce987b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac40318d0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 -- 172.21.15.19:0/1102021184 shutdown_connections 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x558e9ceac6f0 0x558e9ceadb60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 0x558e9ce98cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce912f0 0x558e9ce987b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 --2- 172.21.15.19:0/1102021184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce909d0 0x558e9ce9b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffaaffff640 0 client.15255 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:06:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 -- 172.21.15.19:0/1102021184 >> 172.21.15.19:0/1102021184 conn(0x558e9ce8cf60 msgr2=0x558e9ce92c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 -- 172.21.15.19:0/1102021184 shutdown_connections 2024-09-19T04:06:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 1 -- 172.21.15.19:0/1102021184 wait complete. 2024-09-19T04:06:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.067+0000 7ffad927a740 20 client.15255 trim_cache size 0 max 16384 2024-09-19T04:06:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffad927a740 1 Processor -- start 2024-09-19T04:06:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffad927a740 1 -- start start 2024-09-19T04:06:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffad927a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce909d0 0x558e9ceadb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffad927a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 0x558e9ce93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffad927a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 0x558e9ce8b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffad927a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558e9ceabea0 con 0x558e9ce909d0 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffad927a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558e9ceac570 con 0x558e9ce912f0 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffad927a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e9ceacc40 con 0x558e9ce9bbb0 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffacffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce909d0 0x558e9ceadb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffacffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce909d0 0x558e9ceadb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46648/0 (socket says 172.21.15.19:46648) 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffacffff640 1 -- 172.21.15.19:0/3487725 learned_addr learned my addr 172.21.15.19:0/3487725 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffacffff640 1 -- 172.21.15.19:0/3487725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 msgr2=0x558e9ce8b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffacffff640 1 --2- 172.21.15.19:0/3487725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 0x558e9ce8b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffacffff640 1 -- 172.21.15.19:0/3487725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 msgr2=0x558e9ce93cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffacffff640 1 --2- 172.21.15.19:0/3487725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 0x558e9ce93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.085+0000 7ffacffff640 1 -- 172.21.15.19:0/3487725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9ce9b080 con 0x558e9ce909d0 2024-09-19T04:06:59.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.086+0000 7ffacffff640 1 --2- 172.21.15.19:0/3487725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce909d0 0x558e9ceadb60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffabc014760 tx=0x7ffabc00fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1e7b3d17e65fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.086+0000 7ffaba7fc640 1 -- 172.21.15.19:0/3487725 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc02e070 con 0x558e9ce909d0 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.086+0000 7ffaba7fc640 1 -- 172.21.15.19:0/3487725 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffabc00fcd0 con 0x558e9ce909d0 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.086+0000 7ffaba7fc640 1 -- 172.21.15.19:0/3487725 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc01a070 con 0x558e9ce909d0 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.086+0000 7ffad927a740 1 -- 172.21.15.19:0/3487725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce909d0 msgr2=0x558e9ceadb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.086+0000 7ffad927a740 1 --2- 172.21.15.19:0/3487725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce909d0 0x558e9ceadb60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffabc014760 tx=0x7ffabc00fba0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 -- 172.21.15.19:0/3487725 shutdown_connections 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 --2- 172.21.15.19:0/3487725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce9bbb0 0x558e9ce8b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 --2- 172.21.15.19:0/3487725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 0x558e9ce93cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 --2- 172.21.15.19:0/3487725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce909d0 0x558e9ceadb60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 -- 172.21.15.19:0/3487725 >> 172.21.15.19:0/3487725 conn(0x558e9ce8cf60 msgr2=0x558e9ce935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:59.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 -- 172.21.15.19:0/3487725 shutdown_connections 2024-09-19T04:06:59.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 -- 172.21.15.19:0/3487725 wait complete. 2024-09-19T04:06:59.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 -- start start 2024-09-19T04:06:59.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 Processor -- start 2024-09-19T04:06:59.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce909d0 0x558e9ceb5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 0x558e9ceb6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ceaf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558e9cead9e0 con 0x558e9ce9bbb0 2024-09-19T04:06:59.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558e9cead310 con 0x558e9ce912f0 2024-09-19T04:06:59.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.087+0000 7ffad927a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e9ceae0b0 con 0x558e9ce909d0 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffad927a740 10 client.0 _ll_register_callbacks cb 0x7ffac4005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ceaf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ceaf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46654/0 (socket says 172.21.15.19:46654) 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 -- 172.21.15.19:0/1705802241 learned_addr learned my addr 172.21.15.19:0/1705802241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 0x558e9ceb6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce909d0 0x558e9ceb5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 -- 172.21.15.19:0/1705802241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce909d0 msgr2=0x558e9ceb5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce909d0 0x558e9ceb5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 -- 172.21.15.19:0/1705802241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 msgr2=0x558e9ceb6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 0x558e9ceb6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 -- 172.21.15.19:0/1705802241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9ceafbc0 con 0x558e9ce9bbb0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffacffff640 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ceaf3e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00cb90 tx=0x7ffabc005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539563dd9b4a92dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffaaffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffaaffff640 1 -- 172.21.15.19:0/1705802241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc02e030 con 0x558e9ce9bbb0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffaaffff640 1 -- 172.21.15.19:0/1705802241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7ffabc0053e0 con 0x558e9ce9bbb0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.088+0000 7ffad927a740 1 -- 172.21.15.19:0/1705802241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffabc00fcd0 con 0x558e9ce9bbb0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffaaffff640 1 -- 172.21.15.19:0/1705802241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffabc040050 con 0x558e9ce9bbb0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad927a740 10 client.15261 fetch_fsmap learned FSMap version 28 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad927a740 1 -- 172.21.15.19:0/1705802241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e9ceb45a0 con 0x558e9ce9bbb0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffaaffff640 1 -- 172.21.15.19:0/1705802241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffabc00fcd0 con 0x558e9ce9bbb0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffaaffff640 1 -- 172.21.15.19:0/1705802241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffabc004c20 con 0x558e9ce9bbb0 2024-09-19T04:06:59.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad927a740 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad927a740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad927a740 20 client.15261 populate_metadata read hostname 'smithi019' 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad927a740 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x558e9cf1ac30 0x558e9cf19940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad927a740 4 client.15261 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad927a740 1 -- 172.21.15.19:0/1705802241 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "status"}) -- 0x558e9cdfeb10 con 0x558e9cf1ac30 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.089+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x558e9cf1ac30 0x558e9cf19940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad4c3d640 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x558e9cf1ac30 0x558e9cf19940 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df500df7844d673e server_cookie=960e699b40377eea in_seq=0 out_seq=0 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffaaffff640 10 client.15261 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffaaffff640 1 -- 172.21.15.19:0/1705802241 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558e9cdfeb10 con 0x558e9cf1ac30 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffaaffff640 10 client.15261 handle_command_reply: tid=0 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 client.15261 shutdown 2024-09-19T04:06:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 -- 172.21.15.19:0/1705802241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 msgr2=0x558e9ceaf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ceaf3e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00cb90 tx=0x7ffabc005240 comp rx=0 tx=0).stop 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 -- 172.21.15.19:0/1705802241 shutdown_connections 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558e9ce9bbb0 0x558e9ceaf3e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e9ce912f0 0x558e9ceb6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x558e9cf1ac30 0x558e9cf19940 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 --2- 172.21.15.19:0/1705802241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e9ce909d0 0x558e9ceb5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 -- 172.21.15.19:0/1705802241 >> 172.21.15.19:0/1705802241 conn(0x558e9ce8cf60 msgr2=0x558e9ce8c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffaaffff640 0 client.15261 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 -- 172.21.15.19:0/1705802241 shutdown_connections 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 1 -- 172.21.15.19:0/1705802241 wait complete. 2024-09-19T04:06:59.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.090+0000 7ffad927a740 20 client.15261 trim_cache size 0 max 16384 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24476, 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 138.786883945, 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-19T04:06:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:06:59.100 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-09-19T04:06:59.100 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 149.32997613200001 2024-09-19T04:06:59.100 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:06:59.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 -- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a3240 msgr2=0x7ffac00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a3240 0x7ffac00a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffac8098ba0 tx=0x7ffac80916b0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 -- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0106f00 msgr2=0x7ffad019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0106f00 0x7ffad019cd50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffac800eaa0 tx=0x7ffac800ef60 comp rx=0 tx=0).stop 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 -- 172.21.15.19:0/1281127504 shutdown_connections 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a3240 0x7ffac00a5700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0106f00 0x7ffad019cd50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0102780 0x7ffad01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 --2- 172.21.15.19:0/1281127504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad0101dd0 0x7ffad0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 -- 172.21.15.19:0/1281127504 >> 172.21.15.19:0/1281127504 conn(0x7ffad00fd890 msgr2=0x7ffad00fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 -- 172.21.15.19:0/1281127504 shutdown_connections 2024-09-19T04:06:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:06:59.091+0000 7ffad6ea1640 1 -- 172.21.15.19:0/1281127504 wait complete. 2024-09-19T04:06:59.851 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T04:06:59.861 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T04:06:59.861 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T04:06:59.862 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T04:06:59.862 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T04:06:59.887 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T04:06:59.887 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T04:06:59.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:06:59.888 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-09-19T04:06:59.905 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-19T04:06:59.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:06:59.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:06:59.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:06:59.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-09-19T04:07:00.067 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-19T04:07:00.068 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:07:00.068 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:07:00.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T04:07:00.088 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:06:59 smithi019 ceph-mon[30938]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:07:00.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:06:59 smithi196 ceph-mon[35731]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:07:00.327 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:06:59 smithi111 ceph-mon[35365]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T04:07:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 -- 172.21.15.19:0/559676380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880106a80 msgr2=0x7fd880106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 --2- 172.21.15.19:0/559676380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880106a80 0x7fd880106e60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd870009930 tx=0x7fd87002a420 comp rx=0 tx=0).stop 2024-09-19T04:07:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 -- 172.21.15.19:0/559676380 shutdown_connections 2024-09-19T04:07:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 --2- 172.21.15.19:0/559676380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880101420 0x7fd88010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 --2- 172.21.15.19:0/559676380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd880100a80 0x7fd880100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 --2- 172.21.15.19:0/559676380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880106a80 0x7fd880106e60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 -- 172.21.15.19:0/559676380 >> 172.21.15.19:0/559676380 conn(0x7fd8800fc7a0 msgr2=0x7fd8800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 -- 172.21.15.19:0/559676380 shutdown_connections 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 -- 172.21.15.19:0/559676380 wait complete. 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.783+0000 7fd8864aa640 1 Processor -- start 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd8864aa640 1 -- start start 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd8864aa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd880100a80 0x7fd880102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd8864aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880101420 0x7fd880102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd8864aa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880106a80 0x7fd8801030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd8864aa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd880110e80 con 0x7fd880101420 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd8864aa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd880110d00 con 0x7fd880100a80 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd8864aa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd880111000 con 0x7fd880106a80 2024-09-19T04:07:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880101420 0x7fd880102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880101420 0x7fd880102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46670/0 (socket says 172.21.15.19:46670) 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87f7fe640 1 -- 172.21.15.19:0/1410517169 learned_addr learned my addr 172.21.15.19:0/1410517169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd884a47640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880106a80 0x7fd8801030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87ffff640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd880100a80 0x7fd880102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87f7fe640 1 -- 172.21.15.19:0/1410517169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880106a80 msgr2=0x7fd8801030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87f7fe640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880106a80 0x7fd8801030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87f7fe640 1 -- 172.21.15.19:0/1410517169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd880100a80 msgr2=0x7fd880102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87f7fe640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd880100a80 0x7fd880102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.784+0000 7fd87f7fe640 1 -- 172.21.15.19:0/1410517169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8801a6890 con 0x7fd880101420 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.785+0000 7fd87f7fe640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880101420 0x7fd880102b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd87400d950 tx=0x7fd87400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.785+0000 7fd87d7fa640 1 -- 172.21.15.19:0/1410517169 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874017030 con 0x7fd880101420 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.785+0000 7fd87d7fa640 1 -- 172.21.15.19:0/1410517169 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd874004020 con 0x7fd880101420 2024-09-19T04:07:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.785+0000 7fd87d7fa640 1 -- 172.21.15.19:0/1410517169 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8740047a0 con 0x7fd880101420 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.785+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8801a6b80 con 0x7fd880101420 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.785+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8801a7110 con 0x7fd880101420 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.786+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd880100ee0 con 0x7fd880101420 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.787+0000 7fd87d7fa640 1 -- 172.21.15.19:0/1410517169 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd8740041d0 con 0x7fd880101420 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.788+0000 7fd87d7fa640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8680a32c0 0x7fd8680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.788+0000 7fd87d7fa640 1 -- 172.21.15.19:0/1410517169 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fd8740cfdb0 con 0x7fd880101420 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.788+0000 7fd87ffff640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8680a32c0 0x7fd8680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.789+0000 7fd87ffff640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8680a32c0 0x7fd8680a5780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd870005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:00.791+0000 7fd87d7fa640 1 -- 172.21.15.19:0/1410517169 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd87409c750 con 0x7fd880101420 2024-09-19T04:07:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.010+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd880103b80 con 0x7fd880101420 2024-09-19T04:07:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.010+0000 7fd87d7fa640 1 -- 172.21.15.19:0/1410517169 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7fd880103b80 con 0x7fd880101420 2024-09-19T04:07:01.012 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T04:07:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8680a32c0 msgr2=0x7fd8680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8680a32c0 0x7fd8680a5780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd870005a20 comp rx=0 tx=0).stop 2024-09-19T04:07:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880101420 msgr2=0x7fd880102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880101420 0x7fd880102b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd87400d950 tx=0x7fd87400de10 comp rx=0 tx=0).stop 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 shutdown_connections 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8680a32c0 0x7fd8680a5780 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880106a80 0x7fd8801030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880101420 0x7fd880102b90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 --2- 172.21.15.19:0/1410517169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd880100a80 0x7fd880102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 >> 172.21.15.19:0/1410517169 conn(0x7fd8800fc7a0 msgr2=0x7fd88006a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 shutdown_connections 2024-09-19T04:07:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01.013+0000 7fd8864aa640 1 -- 172.21.15.19:0/1410517169 wait complete. 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:07:01.685 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:07:01.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:07:01.686 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-09-19T04:07:01.744 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:07:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:07:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:07:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:07:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-09-19T04:07:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:07:01.744 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:07:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42149sec preferred_lft 42149sec 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T04:07:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-09-19T04:07:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-09-19T04:07:01.746 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T04:07:01.746 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:07:01.746 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:07:01.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-09-19T04:07:01.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-09-19T04:07:01.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-09-19T04:07:01.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T04:07:01.746 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:07:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:07:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:01.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:07:01.892 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:07:01.918 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-09-19T04:07:01.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:07:01.920 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-09-19T04:07:01.979 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-09-19T04:07:01.979 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:07:01.979 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T04:07:01.979 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T04:07:01.979 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:07:01.980 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:07:01.980 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:07:01.980 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T04:07:01.980 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T04:07:01.980 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T04:07:01.980 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:07:02.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:07:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:02.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:07:02.108 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-09-19T04:07:02.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:07:02.124 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-09-19T04:07:02.180 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:07:02.180 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:07:02.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:07:02.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T04:07:02.180 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:07:02.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:01 smithi019 ceph-mon[30938]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:07:02.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:01 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1410517169' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:07:02.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:01 smithi196 ceph-mon[35731]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:07:02.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:01 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1410517169' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:07:02.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:07:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:02.260 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T04:07:02.260 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:07:02.260 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:07:02.260 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T04:07:02.260 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T04:07:02.260 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T04:07:02.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T04:07:02.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T04:07:02.261 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:07:02.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:07:02.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:02.380 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:07:02.381 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-19T04:07:02.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-09-19T04:07:02.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T04:07:02.381 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-19T04:07:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:01 smithi111 ceph-mon[35365]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T04:07:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:01 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1410517169' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T04:07:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:07:02.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:02.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:07:02.461 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:07:02.477 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:07:02.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:07:02.478 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:07:02.533 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T04:07:02.533 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T04:07:02.533 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/ec843e81-1751-4714-98f4-ed467c5abb98 2024-09-19T04:07:02.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:07:02.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/ec843e81-1751-4714-98f4-ed467c5abb98 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-19T04:07:02.618 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-19T04:07:02.618 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-19T04:07:02.618 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-09-19T04:07:02.775 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-19T04:07:02.775 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-19T04:07:02.776 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-09-19T04:07:02.776 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.111:3300,172.21.15.196:3300" 2024-09-19T04:07:02.776 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:3300,172.21.15.111:3300,172.21.15.196:3300:/volumes/qa/sv_1/ec843e81-1751-4714-98f4-ed467c5abb98 2024-09-19T04:07:02.776 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=0518de98-763b-11ef-bceb-c7b262605968" will pass to kernel 2024-09-19T04:07:02.777 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T04:07:02.777 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-09-19T04:07:02.777 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T04:07:02.777 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T04:07:02.777 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T04:07:02.779 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T04:07:02.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:07:02.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:07:02.806 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:07:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:07:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:02.905 INFO:teuthology.orchestra.run.smithi019.stdout:client24904 2024-09-19T04:07:02.905 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T04:07:02.905 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:07:02.923 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:07:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:07:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:03.020 INFO:teuthology.orchestra.run.smithi019.stdout:client24904 2024-09-19T04:07:03.021 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T04:07:03.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:07:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:03.078 INFO:teuthology.orchestra.run.smithi019.stdout:client24904 2024-09-19T04:07:03.079 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0518de98-763b-11ef-bceb-c7b262605968.client24904/status') 2024-09-19T04:07:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:07:03.139 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.24904 (3)172.21.15.19:0/4070232516 2024-09-19T04:07:03.139 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-09-19T04:07:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-09-19T04:07:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-09-19T04:07:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:71 bytes copied, 2.4056e-05 s, 3.0 MB/s 2024-09-19T04:07:03.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:07:03.146 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T04:07:03.156 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T04:07:03.156 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T04:07:03.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T04:07:03.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 -- 172.21.15.19:0/717461329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 msgr2=0x7f23b4109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:03.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 --2- 172.21.15.19:0/717461329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 0x7f23b4109ca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009ca0 tx=0x7f23ac02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 -- 172.21.15.19:0/717461329 shutdown_connections 2024-09-19T04:07:03.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 --2- 172.21.15.19:0/717461329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 0x7f23b4109ca0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 --2- 172.21.15.19:0/717461329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b4104f80 0x7f23b4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 --2- 172.21.15.19:0/717461329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b4102650 0x7f23b4104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 -- 172.21.15.19:0/717461329 >> 172.21.15.19:0/717461329 conn(0x7f23b40fc1b0 msgr2=0x7f23b40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:03.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 -- 172.21.15.19:0/717461329 shutdown_connections 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 -- 172.21.15.19:0/717461329 wait complete. 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.261+0000 7f23b9653640 1 Processor -- start 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b9653640 1 -- start start 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b9653640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b4102650 0x7f23b4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b9653640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b4104f80 0x7f23b4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b9653640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 0x7f23b419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b9653640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23b410d810 con 0x7f23b41078b0 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b9653640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23b410d690 con 0x7f23b4104f80 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b9653640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23b410d990 con 0x7f23b4102650 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 0x7f23b419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 0x7f23b419f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38868/0 (socket says 172.21.15.19:38868) 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 -- 172.21.15.19:0/751325495 learned_addr learned my addr 172.21.15.19:0/751325495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:03.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b27fc640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b4104f80 0x7f23b4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b2ffd640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b4102650 0x7f23b4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 -- 172.21.15.19:0/751325495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b4102650 msgr2=0x7f23b4197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b4102650 0x7f23b4197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 -- 172.21.15.19:0/751325495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b4104f80 msgr2=0x7f23b4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b4104f80 0x7f23b4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 -- 172.21.15.19:0/751325495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b419fa10 con 0x7f23b41078b0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b2ffd640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b4102650 0x7f23b4197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.262+0000 7f23b37fe640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 0x7f23b419f3a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f23ac02f540 tx=0x7f23ac031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.263+0000 7f239bfff640 1 -- 172.21.15.19:0/751325495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac003c30 con 0x7f23b41078b0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.263+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b419fca0 con 0x7f23b41078b0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.263+0000 7f239bfff640 1 -- 172.21.15.19:0/751325495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ac003dd0 con 0x7f23b41078b0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.263+0000 7f239bfff640 1 -- 172.21.15.19:0/751325495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac02fd90 con 0x7f23b41078b0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.263+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b41a01e0 con 0x7f23b41078b0 2024-09-19T04:07:03.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.264+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b40fdca0 con 0x7f23b41078b0 2024-09-19T04:07:03.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.265+0000 7f239bfff640 1 -- 172.21.15.19:0/751325495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f23ac004050 con 0x7f23b41078b0 2024-09-19T04:07:03.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.265+0000 7f239bfff640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f239c0a32f0 0x7f239c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:03.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.266+0000 7f23b2ffd640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f239c0a32f0 0x7f239c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:03.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.266+0000 7f239bfff640 1 -- 172.21.15.19:0/751325495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f23ac0f0e60 con 0x7f23b41078b0 2024-09-19T04:07:03.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.266+0000 7f23b2ffd640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f239c0a32f0 0x7f239c0a57b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23a4005d00 tx=0x7f23a4005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:03.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.270+0000 7f239bfff640 1 -- 172.21.15.19:0/751325495 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f23ac0bd900 con 0x7f23b41078b0 2024-09-19T04:07:03.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.492+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f23b4062df0 con 0x7f23b41078b0 2024-09-19T04:07:03.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.955+0000 7f239bfff640 1 -- 172.21.15.19:0/751325495 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f23ac0bd200 con 0x7f23b41078b0 2024-09-19T04:07:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f239c0a32f0 msgr2=0x7f239c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f239c0a32f0 0x7f239c0a57b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23a4005d00 tx=0x7f23a4005c70 comp rx=0 tx=0).stop 2024-09-19T04:07:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 msgr2=0x7f23b419f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 0x7f23b419f3a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f23ac02f540 tx=0x7f23ac031ee0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 shutdown_connections 2024-09-19T04:07:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f239c0a32f0 0x7f239c0a57b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23b41078b0 0x7f23b419f3a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b4104f80 0x7f23b4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 --2- 172.21.15.19:0/751325495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b4102650 0x7f23b4197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 >> 172.21.15.19:0/751325495 conn(0x7f23b40fc1b0 msgr2=0x7f23b4102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 shutdown_connections 2024-09-19T04:07:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:03.958+0000 7f23b9653640 1 -- 172.21.15.19:0/751325495 wait complete. 2024-09-19T04:07:03.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T04:07:04.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 -- 172.21.15.19:0/2458608814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 msgr2=0x7f54e01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:04.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 --2- 172.21.15.19:0/2458608814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 0x7f54e01020d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009900 tx=0x7f54d002a410 comp rx=0 tx=0).stop 2024-09-19T04:07:04.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 -- 172.21.15.19:0/2458608814 shutdown_connections 2024-09-19T04:07:04.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 --2- 172.21.15.19:0/2458608814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e0102610 0x7f54e0108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 --2- 172.21.15.19:0/2458608814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 0x7f54e01020d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 --2- 172.21.15.19:0/2458608814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54e0100a60 0x7f54e0100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 -- 172.21.15.19:0/2458608814 >> 172.21.15.19:0/2458608814 conn(0x7f54e00fc1e0 msgr2=0x7f54e00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 -- 172.21.15.19:0/2458608814 shutdown_connections 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 -- 172.21.15.19:0/2458608814 wait complete. 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 Processor -- start 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.086+0000 7f54e8331640 1 -- start start 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e8331640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e0100a60 0x7f54e0196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e8331640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 0x7f54e0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e8331640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54e0102610 0x7f54e019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e8331640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010c710 con 0x7f54e0101c50 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e8331640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010c590 con 0x7f54e0102610 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e8331640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010c890 con 0x7f54e0100a60 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 0x7f54e0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 0x7f54e0197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38888/0 (socket says 172.21.15.19:38888) 2024-09-19T04:07:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 -- 172.21.15.19:0/3216855982 learned_addr learned my addr 172.21.15.19:0/3216855982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e60cd640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e0100a60 0x7f54e0196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e68ce640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54e0102610 0x7f54e019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 -- 172.21.15.19:0/3216855982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e0100a60 msgr2=0x7f54e0196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e0100a60 0x7f54e0196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 -- 172.21.15.19:0/3216855982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54e0102610 msgr2=0x7f54e019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54e0102610 0x7f54e019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 -- 172.21.15.19:0/3216855982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e019e930 con 0x7f54e0101c50 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.087+0000 7f54e58cc640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 0x7f54e0197260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f54d0031440 tx=0x7f54d0032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.088+0000 7f54d77fe640 1 -- 172.21.15.19:0/3216855982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d003f030 con 0x7f54e0101c50 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.088+0000 7f54d77fe640 1 -- 172.21.15.19:0/3216855982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d0003ce0 con 0x7f54e0101c50 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.088+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e019ebc0 con 0x7f54e0101c50 2024-09-19T04:07:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.088+0000 7f54d77fe640 1 -- 172.21.15.19:0/3216855982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0035070 con 0x7f54e0101c50 2024-09-19T04:07:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.088+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e019f0b0 con 0x7f54e0101c50 2024-09-19T04:07:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.089+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e0107dd0 con 0x7f54e0101c50 2024-09-19T04:07:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.090+0000 7f54d77fe640 1 -- 172.21.15.19:0/3216855982 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f54d0042030 con 0x7f54e0101c50 2024-09-19T04:07:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.090+0000 7f54d77fe640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f54cc0a32f0 0x7f54cc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.091+0000 7f54e60cd640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f54cc0a32f0 0x7f54cc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.091+0000 7f54d77fe640 1 -- 172.21.15.19:0/3216855982 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f54d00f0330 con 0x7f54e0101c50 2024-09-19T04:07:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.091+0000 7f54e60cd640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f54cc0a32f0 0x7f54cc0a57b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f54e01981f0 tx=0x7f54dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:04.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.095+0000 7f54d77fe640 1 -- 172.21.15.19:0/3216855982 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f54d00bc600 con 0x7f54e0101c50 2024-09-19T04:07:04.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:03 smithi196 ceph-mon[35731]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:07:04.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:03 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/751325495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T04:07:04.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.298+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f54e0062790 con 0x7f54cc0a32f0 2024-09-19T04:07:04.305 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:03 smithi019 ceph-mon[30938]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:07:04.305 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:03 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/751325495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T04:07:04.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.312+0000 7f54d77fe640 1 -- 172.21.15.19:0/3216855982 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f54e0062790 con 0x7f54cc0a32f0 2024-09-19T04:07:04.313 INFO:teuthology.orchestra.run.smithi019.stdout:{} 2024-09-19T04:07:04.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f54cc0a32f0 msgr2=0x7f54cc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:04.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f54cc0a32f0 0x7f54cc0a57b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f54e01981f0 tx=0x7f54dc00b390 comp rx=0 tx=0).stop 2024-09-19T04:07:04.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 msgr2=0x7f54e0197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:04.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 0x7f54e0197260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f54d0031440 tx=0x7f54d0032e20 comp rx=0 tx=0).stop 2024-09-19T04:07:04.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 shutdown_connections 2024-09-19T04:07:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f54cc0a32f0 0x7f54cc0a57b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54e0102610 0x7f54e019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e0101c50 0x7f54e0197260 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 --2- 172.21.15.19:0/3216855982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e0100a60 0x7f54e0196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 >> 172.21.15.19:0/3216855982 conn(0x7f54e00fc1e0 msgr2=0x7f54e0104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 shutdown_connections 2024-09-19T04:07:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:04.314+0000 7f54e8331640 1 -- 172.21.15.19:0/3216855982 wait complete. 2024-09-19T04:07:04.326 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T04:07:04.355 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-19T04:07:04.375 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T04:07:04.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-19T04:07:04.378 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T04:07:04.434 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T04:07:04.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:07:04.435 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:03 smithi111 ceph-mon[35365]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:07:04.435 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:03 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/751325495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T04:07:04.681 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:07:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:07:04.303+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:07:04.682 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:07:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:07:04.303+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:07:04.682 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:07:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:07:04.303+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:07:04.682 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:07:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:07:04.303+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:07:04.683 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:07:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:07:04.303+0000 7f06b5569640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T04:07:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:05.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:04 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/751325495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T04:07:05.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:04 smithi019 ceph-mon[30938]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:07:05.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:04 smithi019 ceph-mon[30938]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:05.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:04 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/751325495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T04:07:05.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:04 smithi196 ceph-mon[35731]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:07:05.208 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:04 smithi196 ceph-mon[35731]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/814980806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8102570 msgr2=0x7fd7d81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/814980806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8102570 0x7fd7d81029b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009900 tx=0x7fd7cc02a480 comp rx=0 tx=0).stop 2024-09-19T04:07:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/814980806 shutdown_connections 2024-09-19T04:07:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/814980806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d8102ef0 0x7fd7d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/814980806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8102570 0x7fd7d81029b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/814980806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8100e20 0x7fd7d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/814980806 >> 172.21.15.19:0/814980806 conn(0x7fd7d80fc7d0 msgr2=0x7fd7d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/814980806 shutdown_connections 2024-09-19T04:07:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.208+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/814980806 wait complete. 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.209+0000 7fd7dd1b7640 1 Processor -- start 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.209+0000 7fd7dd1b7640 1 -- start start 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.209+0000 7fd7dd1b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8100e20 0x7fd7d819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.209+0000 7fd7dd1b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d8102570 0x7fd7d819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.209+0000 7fd7dd1b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8102ef0 0x7fd7d819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.209+0000 7fd7dd1b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d810fe70 con 0x7fd7d8100e20 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.209+0000 7fd7dd1b7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d810fcf0 con 0x7fd7d8102ef0 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.209+0000 7fd7dd1b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d810fff0 con 0x7fd7d8102570 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8100e20 0x7fd7d819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8100e20 0x7fd7d819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38930/0 (socket says 172.21.15.19:38930) 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 -- 172.21.15.19:0/4271747039 learned_addr learned my addr 172.21.15.19:0/4271747039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:05.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d759e640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8102ef0 0x7fd7d819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d659c640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d8102570 0x7fd7d819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 -- 172.21.15.19:0/4271747039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d8102570 msgr2=0x7fd7d819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d8102570 0x7fd7d819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 -- 172.21.15.19:0/4271747039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8102ef0 msgr2=0x7fd7d819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8102ef0 0x7fd7d819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 -- 172.21.15.19:0/4271747039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d819f630 con 0x7fd7d8100e20 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7d6d9d640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8100e20 0x7fd7d819a680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800c920 tx=0x7fd7c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7bffff640 1 -- 172.21.15.19:0/4271747039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8007c70 con 0x7fd7d8100e20 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d819f920 con 0x7fd7d8100e20 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7bffff640 1 -- 172.21.15.19:0/4271747039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c8007e10 con 0x7fd7d8100e20 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.210+0000 7fd7bffff640 1 -- 172.21.15.19:0/4271747039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8018890 con 0x7fd7d8100e20 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.211+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d81a7200 con 0x7fd7d8100e20 2024-09-19T04:07:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.212+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d8101200 con 0x7fd7d8100e20 2024-09-19T04:07:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.213+0000 7fd7bffff640 1 -- 172.21.15.19:0/4271747039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fd7c8018b30 con 0x7fd7d8100e20 2024-09-19T04:07:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.213+0000 7fd7bffff640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7c00a3340 0x7fd7c00a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.214+0000 7fd7d659c640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7c00a3340 0x7fd7c00a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.214+0000 7fd7bffff640 1 -- 172.21.15.19:0/4271747039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fd7c80cfee0 con 0x7fd7d8100e20 2024-09-19T04:07:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.214+0000 7fd7d659c640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7c00a3340 0x7fd7c00a5800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009230 tx=0x7fd7cc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.218+0000 7fd7bffff640 1 -- 172.21.15.19:0/4271747039 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd7c8014030 con 0x7fd7d8100e20 2024-09-19T04:07:05.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.391+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7d8106f80 con 0x7fd7c00a3340 2024-09-19T04:07:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.401+0000 7fd7bffff640 1 -- 172.21.15.19:0/4271747039 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340816 (secure 0 0 0) 0x7fd7d8106f80 con 0x7fd7c00a3340 2024-09-19T04:07:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-19T04:07:05.411 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:04 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/751325495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T04:07:05.411 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:04 smithi111 ceph-mon[35365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T04:07:05.411 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:04 smithi111 ceph-mon[35365]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:07:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7c00a3340 msgr2=0x7fd7c00a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7c00a3340 0x7fd7c00a5800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009230 tx=0x7fd7cc0058d0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8100e20 msgr2=0x7fd7d819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8100e20 0x7fd7d819a680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800c920 tx=0x7fd7c800cde0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 shutdown_connections 2024-09-19T04:07:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd7c00a3340 0x7fd7c00a5800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8102ef0 0x7fd7d819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d8102570 0x7fd7d819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 --2- 172.21.15.19:0/4271747039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d8100e20 0x7fd7d819a680 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 >> 172.21.15.19:0/4271747039 conn(0x7fd7d80fc7d0 msgr2=0x7fd7d8106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 shutdown_connections 2024-09-19T04:07:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:05.406+0000 7fd7dd1b7640 1 -- 172.21.15.19:0/4271747039 wait complete. 2024-09-19T04:07:06.102 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-19T04:07:04.444927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660191,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1161,"num_read_kb":708,"num_write":439,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1241696,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349440,"kb_used_data":12600,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592384,"statfs":{"total":1151940427776,"available":1151582601216,"internally_reserved":0,"allocated":12902400,"data_stored":8297039,"data_compressed":470438,"data_compressed_allocated":638976,"data_compressed_original":2809856,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":70,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002872"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:01.739932+0000","last_change":"2024-09-19T04:04:37.214769+0000","last_active":"2024-09-19T04:07:01.739932+0000","last_peered":"2024-09-19T04:07:01.739932+0000","last_clean":"2024-09-19T04:07:01.739932+0000","last_became_active":"2024-09-19T04:04:37.214518+0000","last_became_peered":"2024-09-19T04:04:37.214518+0000","last_unstale":"2024-09-19T04:07:01.739932+0000","last_undegraded":"2024-09-19T04:07:01.739932+0000","last_fullsized":"2024-09-19T04:07:01.739932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:50.704566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294660+0000","last_change":"2024-09-19T04:04:39.329733+0000","last_active":"2024-09-19T04:06:43.294660+0000","last_peered":"2024-09-19T04:06:43.294660+0000","last_clean":"2024-09-19T04:06:43.294660+0000","last_became_active":"2024-09-19T04:04:39.329031+0000","last_became_peered":"2024-09-19T04:04:39.329031+0000","last_unstale":"2024-09-19T04:06:43.294660+0000","last_undegraded":"2024-09-19T04:06:43.294660+0000","last_fullsized":"2024-09-19T04:06:43.294660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:57.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913948+0000","last_change":"2024-09-19T04:04:46.362679+0000","last_active":"2024-09-19T04:06:42.913948+0000","last_peered":"2024-09-19T04:06:42.913948+0000","last_clean":"2024-09-19T04:06:42.913948+0000","last_became_active":"2024-09-19T04:04:46.362257+0000","last_became_peered":"2024-09-19T04:04:46.362257+0000","last_unstale":"2024-09-19T04:06:42.913948+0000","last_undegraded":"2024-09-19T04:06:42.913948+0000","last_fullsized":"2024-09-19T04:06:42.913948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.389839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294777+0000","last_change":"2024-09-19T04:04:37.647295+0000","last_active":"2024-09-19T04:06:43.294777+0000","last_peered":"2024-09-19T04:06:43.294777+0000","last_clean":"2024-09-19T04:06:43.294777+0000","last_became_active":"2024-09-19T04:04:37.646964+0000","last_became_peered":"2024-09-19T04:04:37.646964+0000","last_unstale":"2024-09-19T04:06:43.294777+0000","last_undegraded":"2024-09-19T04:06:43.294777+0000","last_fullsized":"2024-09-19T04:06:43.294777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:49.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913163+0000","last_change":"2024-09-19T04:04:39.318270+0000","last_active":"2024-09-19T04:06:42.913163+0000","last_peered":"2024-09-19T04:06:42.913163+0000","last_clean":"2024-09-19T04:06:42.913163+0000","last_became_active":"2024-09-19T04:04:39.317972+0000","last_became_peered":"2024-09-19T04:04:39.317972+0000","last_unstale":"2024-09-19T04:06:42.913163+0000","last_undegraded":"2024-09-19T04:06:42.913163+0000","last_fullsized":"2024-09-19T04:06:42.913163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:25.994112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914875+0000","last_change":"2024-09-19T04:04:46.355111+0000","last_active":"2024-09-19T04:06:42.914875+0000","last_peered":"2024-09-19T04:06:42.914875+0000","last_clean":"2024-09-19T04:06:42.914875+0000","last_became_active":"2024-09-19T04:04:46.354684+0000","last_became_peered":"2024-09-19T04:04:46.354684+0000","last_unstale":"2024-09-19T04:06:42.914875+0000","last_undegraded":"2024-09-19T04:06:42.914875+0000","last_fullsized":"2024-09-19T04:06:42.914875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:33.141690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917054+0000","last_change":"2024-09-19T04:04:37.211772+0000","last_active":"2024-09-19T04:06:42.917054+0000","last_peered":"2024-09-19T04:06:42.917054+0000","last_clean":"2024-09-19T04:06:42.917054+0000","last_became_active":"2024-09-19T04:04:37.211452+0000","last_became_peered":"2024-09-19T04:04:37.211452+0000","last_unstale":"2024-09-19T04:06:42.917054+0000","last_undegraded":"2024-09-19T04:06:42.917054+0000","last_fullsized":"2024-09-19T04:06:42.917054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:57.781481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110814+0000","last_change":"2024-09-19T04:04:39.320791+0000","last_active":"2024-09-19T04:06:44.110814+0000","last_peered":"2024-09-19T04:06:44.110814+0000","last_clean":"2024-09-19T04:06:44.110814+0000","last_became_active":"2024-09-19T04:04:39.320206+0000","last_became_peered":"2024-09-19T04:04:39.320206+0000","last_unstale":"2024-09-19T04:06:44.110814+0000","last_undegraded":"2024-09-19T04:06:44.110814+0000","last_fullsized":"2024-09-19T04:06:44.110814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:17.385974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915320+0000","last_change":"2024-09-19T04:04:46.363375+0000","last_active":"2024-09-19T04:06:42.915320+0000","last_peered":"2024-09-19T04:06:42.915320+0000","last_clean":"2024-09-19T04:06:42.915320+0000","last_became_active":"2024-09-19T04:04:46.362800+0000","last_became_peered":"2024-09-19T04:04:46.362800+0000","last_unstale":"2024-09-19T04:06:42.915320+0000","last_undegraded":"2024-09-19T04:06:42.915320+0000","last_fullsized":"2024-09-19T04:06:42.915320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:55.651507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917026+0000","last_change":"2024-09-19T04:04:37.206218+0000","last_active":"2024-09-19T04:06:42.917026+0000","last_peered":"2024-09-19T04:06:42.917026+0000","last_clean":"2024-09-19T04:06:42.917026+0000","last_became_active":"2024-09-19T04:04:37.205675+0000","last_became_peered":"2024-09-19T04:04:37.205675+0000","last_unstale":"2024-09-19T04:06:42.917026+0000","last_undegraded":"2024-09-19T04:06:42.917026+0000","last_fullsized":"2024-09-19T04:06:42.917026+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:27.664960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912735+0000","last_change":"2024-09-19T04:04:39.329667+0000","last_active":"2024-09-19T04:06:42.912735+0000","last_peered":"2024-09-19T04:06:42.912735+0000","last_clean":"2024-09-19T04:06:42.912735+0000","last_became_active":"2024-09-19T04:04:39.328180+0000","last_became_peered":"2024-09-19T04:04:39.328180+0000","last_unstale":"2024-09-19T04:06:42.912735+0000","last_undegraded":"2024-09-19T04:06:42.912735+0000","last_fullsized":"2024-09-19T04:06:42.912735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:59.782472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914994+0000","last_change":"2024-09-19T04:04:46.359432+0000","last_active":"2024-09-19T04:06:42.914994+0000","last_peered":"2024-09-19T04:06:42.914994+0000","last_clean":"2024-09-19T04:06:42.914994+0000","last_became_active":"2024-09-19T04:04:46.358966+0000","last_became_peered":"2024-09-19T04:04:46.358966+0000","last_unstale":"2024-09-19T04:06:42.914994+0000","last_undegraded":"2024-09-19T04:06:42.914994+0000","last_fullsized":"2024-09-19T04:06:42.914994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:41.537926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418761+0000","last_change":"2024-09-19T04:04:39.325084+0000","last_active":"2024-09-19T04:06:43.418761+0000","last_peered":"2024-09-19T04:06:43.418761+0000","last_clean":"2024-09-19T04:06:43.418761+0000","last_became_active":"2024-09-19T04:04:39.324126+0000","last_became_peered":"2024-09-19T04:04:39.324126+0000","last_unstale":"2024-09-19T04:06:43.418761+0000","last_undegraded":"2024-09-19T04:06:43.418761+0000","last_fullsized":"2024-09-19T04:06:43.418761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:13.523889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110894+0000","last_change":"2024-09-19T04:05:42.359098+0000","last_active":"2024-09-19T04:06:44.110894+0000","last_peered":"2024-09-19T04:06:44.110894+0000","last_clean":"2024-09-19T04:06:44.110894+0000","last_became_active":"2024-09-19T04:05:42.341384+0000","last_became_peered":"2024-09-19T04:05:42.341384+0000","last_unstale":"2024-09-19T04:06:44.110894+0000","last_undegraded":"2024-09-19T04:06:44.110894+0000","last_fullsized":"2024-09-19T04:06:44.110894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:15:57.418355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914959+0000","last_change":"2024-09-19T04:04:46.374921+0000","last_active":"2024-09-19T04:06:42.914959+0000","last_peered":"2024-09-19T04:06:42.914959+0000","last_clean":"2024-09-19T04:06:42.914959+0000","last_became_active":"2024-09-19T04:04:46.374466+0000","last_became_peered":"2024-09-19T04:04:46.374466+0000","last_unstale":"2024-09-19T04:06:42.914959+0000","last_undegraded":"2024-09-19T04:06:42.914959+0000","last_fullsized":"2024-09-19T04:06:42.914959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:46.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294259+0000","last_change":"2024-09-19T04:04:39.319948+0000","last_active":"2024-09-19T04:06:43.294259+0000","last_peered":"2024-09-19T04:06:43.294259+0000","last_clean":"2024-09-19T04:06:43.294259+0000","last_became_active":"2024-09-19T04:04:39.319675+0000","last_became_peered":"2024-09-19T04:04:39.319675+0000","last_unstale":"2024-09-19T04:06:43.294259+0000","last_undegraded":"2024-09-19T04:06:43.294259+0000","last_fullsized":"2024-09-19T04:06:43.294259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:08.818133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914433+0000","last_change":"2024-09-19T04:04:37.207705+0000","last_active":"2024-09-19T04:06:42.914433+0000","last_peered":"2024-09-19T04:06:42.914433+0000","last_clean":"2024-09-19T04:06:42.914433+0000","last_became_active":"2024-09-19T04:04:37.207251+0000","last_became_peered":"2024-09-19T04:04:37.207251+0000","last_unstale":"2024-09-19T04:06:42.914433+0000","last_undegraded":"2024-09-19T04:06:42.914433+0000","last_fullsized":"2024-09-19T04:06:42.914433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:20.835004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914870+0000","last_change":"2024-09-19T04:04:46.354463+0000","last_active":"2024-09-19T04:06:42.914870+0000","last_peered":"2024-09-19T04:06:42.914870+0000","last_clean":"2024-09-19T04:06:42.914870+0000","last_became_active":"2024-09-19T04:04:46.353754+0000","last_became_peered":"2024-09-19T04:04:46.353754+0000","last_unstale":"2024-09-19T04:06:42.914870+0000","last_undegraded":"2024-09-19T04:06:42.914870+0000","last_fullsized":"2024-09-19T04:06:42.914870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:09.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110995+0000","last_change":"2024-09-19T04:04:39.322727+0000","last_active":"2024-09-19T04:06:44.110995+0000","last_peered":"2024-09-19T04:06:44.110995+0000","last_clean":"2024-09-19T04:06:44.110995+0000","last_became_active":"2024-09-19T04:04:39.320989+0000","last_became_peered":"2024-09-19T04:04:39.320989+0000","last_unstale":"2024-09-19T04:06:44.110995+0000","last_undegraded":"2024-09-19T04:06:44.110995+0000","last_fullsized":"2024-09-19T04:06:44.110995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:06.536540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294398+0000","last_change":"2024-09-19T04:04:37.213602+0000","last_active":"2024-09-19T04:06:43.294398+0000","last_peered":"2024-09-19T04:06:43.294398+0000","last_clean":"2024-09-19T04:06:43.294398+0000","last_became_active":"2024-09-19T04:04:37.213240+0000","last_became_peered":"2024-09-19T04:04:37.213240+0000","last_unstale":"2024-09-19T04:06:43.294398+0000","last_undegraded":"2024-09-19T04:06:43.294398+0000","last_fullsized":"2024-09-19T04:06:43.294398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:09.822393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913644+0000","last_change":"2024-09-19T04:04:46.363921+0000","last_active":"2024-09-19T04:06:42.913644+0000","last_peered":"2024-09-19T04:06:42.913644+0000","last_clean":"2024-09-19T04:06:42.913644+0000","last_became_active":"2024-09-19T04:04:46.363476+0000","last_became_peered":"2024-09-19T04:04:46.363476+0000","last_unstale":"2024-09-19T04:06:42.913644+0000","last_undegraded":"2024-09-19T04:06:42.913644+0000","last_fullsized":"2024-09-19T04:06:42.913644+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:32.015782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917044+0000","last_change":"2024-09-19T04:04:39.316814+0000","last_active":"2024-09-19T04:06:42.917044+0000","last_peered":"2024-09-19T04:06:42.917044+0000","last_clean":"2024-09-19T04:06:42.917044+0000","last_became_active":"2024-09-19T04:04:39.316504+0000","last_became_peered":"2024-09-19T04:04:39.316504+0000","last_unstale":"2024-09-19T04:06:42.917044+0000","last_undegraded":"2024-09-19T04:06:42.917044+0000","last_fullsized":"2024-09-19T04:06:42.917044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110786+0000","last_change":"2024-09-19T04:04:37.647260+0000","last_active":"2024-09-19T04:06:44.110786+0000","last_peered":"2024-09-19T04:06:44.110786+0000","last_clean":"2024-09-19T04:06:44.110786+0000","last_became_active":"2024-09-19T04:04:37.646971+0000","last_became_peered":"2024-09-19T04:04:37.646971+0000","last_unstale":"2024-09-19T04:06:44.110786+0000","last_undegraded":"2024-09-19T04:06:44.110786+0000","last_fullsized":"2024-09-19T04:06:44.110786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:07.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915039+0000","last_change":"2024-09-19T04:05:42.342340+0000","last_active":"2024-09-19T04:06:42.915039+0000","last_peered":"2024-09-19T04:06:42.915039+0000","last_clean":"2024-09-19T04:06:42.915039+0000","last_became_active":"2024-09-19T04:05:42.341861+0000","last_became_peered":"2024-09-19T04:05:42.341861+0000","last_unstale":"2024-09-19T04:06:42.915039+0000","last_undegraded":"2024-09-19T04:06:42.915039+0000","last_fullsized":"2024-09-19T04:06:42.915039+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:27.520514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916892+0000","last_change":"2024-09-19T04:06:42.916356+0000","last_active":"2024-09-19T04:06:42.916892+0000","last_peered":"2024-09-19T04:06:42.916892+0000","last_clean":"2024-09-19T04:06:42.916892+0000","last_became_active":"2024-09-19T04:06:42.915926+0000","last_became_peered":"2024-09-19T04:06:42.915926+0000","last_unstale":"2024-09-19T04:06:42.916892+0000","last_undegraded":"2024-09-19T04:06:42.916892+0000","last_fullsized":"2024-09-19T04:06:42.916892+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:46.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915360+0000","last_change":"2024-09-19T04:04:37.703740+0000","last_active":"2024-09-19T04:06:42.915360+0000","last_peered":"2024-09-19T04:06:42.915360+0000","last_clean":"2024-09-19T04:06:42.915360+0000","last_became_active":"2024-09-19T04:04:37.703414+0000","last_became_peered":"2024-09-19T04:04:37.703414+0000","last_unstale":"2024-09-19T04:06:42.915360+0000","last_undegraded":"2024-09-19T04:06:42.915360+0000","last_fullsized":"2024-09-19T04:06:42.915360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:19.653559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914409+0000","last_change":"2024-09-19T04:04:46.354964+0000","last_active":"2024-09-19T04:06:42.914409+0000","last_peered":"2024-09-19T04:06:42.914409+0000","last_clean":"2024-09-19T04:06:42.914409+0000","last_became_active":"2024-09-19T04:04:46.354494+0000","last_became_peered":"2024-09-19T04:04:46.354494+0000","last_unstale":"2024-09-19T04:06:42.914409+0000","last_undegraded":"2024-09-19T04:06:42.914409+0000","last_fullsized":"2024-09-19T04:06:42.914409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:11.246134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294858+0000","last_change":"2024-09-19T04:04:39.315811+0000","last_active":"2024-09-19T04:06:43.294858+0000","last_peered":"2024-09-19T04:06:43.294858+0000","last_clean":"2024-09-19T04:06:43.294858+0000","last_became_active":"2024-09-19T04:04:39.315466+0000","last_became_peered":"2024-09-19T04:04:39.315466+0000","last_unstale":"2024-09-19T04:06:43.294858+0000","last_undegraded":"2024-09-19T04:06:43.294858+0000","last_fullsized":"2024-09-19T04:06:43.294858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:17.414765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913659+0000","last_change":"2024-09-19T04:04:37.204206+0000","last_active":"2024-09-19T04:06:42.913659+0000","last_peered":"2024-09-19T04:06:42.913659+0000","last_clean":"2024-09-19T04:06:42.913659+0000","last_became_active":"2024-09-19T04:04:37.203845+0000","last_became_peered":"2024-09-19T04:04:37.203845+0000","last_unstale":"2024-09-19T04:06:42.913659+0000","last_undegraded":"2024-09-19T04:06:42.913659+0000","last_fullsized":"2024-09-19T04:06:42.913659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:52.868643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.293985+0000","last_change":"2024-09-19T04:04:46.375541+0000","last_active":"2024-09-19T04:06:43.293985+0000","last_peered":"2024-09-19T04:06:43.293985+0000","last_clean":"2024-09-19T04:06:43.293985+0000","last_became_active":"2024-09-19T04:04:46.375142+0000","last_became_peered":"2024-09-19T04:04:46.375142+0000","last_unstale":"2024-09-19T04:06:43.293985+0000","last_undegraded":"2024-09-19T04:06:43.293985+0000","last_fullsized":"2024-09-19T04:06:43.293985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.889459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909494+0000","last_change":"2024-09-19T04:04:39.324094+0000","last_active":"2024-09-19T04:06:42.909494+0000","last_peered":"2024-09-19T04:06:42.909494+0000","last_clean":"2024-09-19T04:06:42.909494+0000","last_became_active":"2024-09-19T04:04:39.322897+0000","last_became_peered":"2024-09-19T04:04:39.322897+0000","last_unstale":"2024-09-19T04:06:42.909494+0000","last_undegraded":"2024-09-19T04:06:42.909494+0000","last_fullsized":"2024-09-19T04:06:42.909494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:07.773110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917125+0000","last_change":"2024-09-19T04:04:37.647686+0000","last_active":"2024-09-19T04:06:42.917125+0000","last_peered":"2024-09-19T04:06:42.917125+0000","last_clean":"2024-09-19T04:06:42.917125+0000","last_became_active":"2024-09-19T04:04:37.646979+0000","last_became_peered":"2024-09-19T04:04:37.646979+0000","last_unstale":"2024-09-19T04:06:42.917125+0000","last_undegraded":"2024-09-19T04:06:42.917125+0000","last_fullsized":"2024-09-19T04:06:42.917125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:48.063075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111034+0000","last_change":"2024-09-19T04:04:46.356078+0000","last_active":"2024-09-19T04:06:44.111034+0000","last_peered":"2024-09-19T04:06:44.111034+0000","last_clean":"2024-09-19T04:06:44.111034+0000","last_became_active":"2024-09-19T04:04:46.355630+0000","last_became_peered":"2024-09-19T04:04:46.355630+0000","last_unstale":"2024-09-19T04:06:44.111034+0000","last_undegraded":"2024-09-19T04:06:44.111034+0000","last_fullsized":"2024-09-19T04:06:44.111034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:45.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914196+0000","last_change":"2024-09-19T04:04:39.311067+0000","last_active":"2024-09-19T04:06:42.914196+0000","last_peered":"2024-09-19T04:06:42.914196+0000","last_clean":"2024-09-19T04:06:42.914196+0000","last_became_active":"2024-09-19T04:04:39.310725+0000","last_became_peered":"2024-09-19T04:04:39.310725+0000","last_unstale":"2024-09-19T04:06:42.914196+0000","last_undegraded":"2024-09-19T04:06:42.914196+0000","last_fullsized":"2024-09-19T04:06:42.914196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:47:11.488121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913434+0000","last_change":"2024-09-19T04:04:37.648335+0000","last_active":"2024-09-19T04:06:42.913434+0000","last_peered":"2024-09-19T04:06:42.913434+0000","last_clean":"2024-09-19T04:06:42.913434+0000","last_became_active":"2024-09-19T04:04:37.648000+0000","last_became_peered":"2024-09-19T04:04:37.648000+0000","last_unstale":"2024-09-19T04:06:42.913434+0000","last_undegraded":"2024-09-19T04:06:42.913434+0000","last_fullsized":"2024-09-19T04:06:42.913434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:16.102728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916512+0000","last_change":"2024-09-19T04:04:46.375653+0000","last_active":"2024-09-19T04:06:42.916512+0000","last_peered":"2024-09-19T04:06:42.916512+0000","last_clean":"2024-09-19T04:06:42.916512+0000","last_became_active":"2024-09-19T04:04:46.375132+0000","last_became_peered":"2024-09-19T04:04:46.375132+0000","last_unstale":"2024-09-19T04:06:42.916512+0000","last_undegraded":"2024-09-19T04:06:42.916512+0000","last_fullsized":"2024-09-19T04:06:42.916512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:03.332697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914089+0000","last_change":"2024-09-19T04:04:39.318155+0000","last_active":"2024-09-19T04:06:42.914089+0000","last_peered":"2024-09-19T04:06:42.914089+0000","last_clean":"2024-09-19T04:06:42.914089+0000","last_became_active":"2024-09-19T04:04:39.316710+0000","last_became_peered":"2024-09-19T04:04:39.316710+0000","last_unstale":"2024-09-19T04:06:42.914089+0000","last_undegraded":"2024-09-19T04:06:42.914089+0000","last_fullsized":"2024-09-19T04:06:42.914089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:59.084523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":147,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:58.736478+0000","last_change":"2024-09-19T04:04:37.210695+0000","last_active":"2024-09-19T04:06:58.736478+0000","last_peered":"2024-09-19T04:06:58.736478+0000","last_clean":"2024-09-19T04:06:58.736478+0000","last_became_active":"2024-09-19T04:04:37.210334+0000","last_became_peered":"2024-09-19T04:04:37.210334+0000","last_unstale":"2024-09-19T04:06:58.736478+0000","last_undegraded":"2024-09-19T04:06:58.736478+0000","last_fullsized":"2024-09-19T04:06:58.736478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:12.545248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909313+0000","last_change":"2024-09-19T04:04:46.358130+0000","last_active":"2024-09-19T04:06:42.909313+0000","last_peered":"2024-09-19T04:06:42.909313+0000","last_clean":"2024-09-19T04:06:42.909313+0000","last_became_active":"2024-09-19T04:04:46.356810+0000","last_became_peered":"2024-09-19T04:04:46.356810+0000","last_unstale":"2024-09-19T04:06:42.909313+0000","last_undegraded":"2024-09-19T04:06:42.909313+0000","last_fullsized":"2024-09-19T04:06:42.909313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:49.884478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916817+0000","last_change":"2024-09-19T04:06:42.916354+0000","last_active":"2024-09-19T04:06:42.916817+0000","last_peered":"2024-09-19T04:06:42.916817+0000","last_clean":"2024-09-19T04:06:42.916817+0000","last_became_active":"2024-09-19T04:06:42.915943+0000","last_became_peered":"2024-09-19T04:06:42.915943+0000","last_unstale":"2024-09-19T04:06:42.916817+0000","last_undegraded":"2024-09-19T04:06:42.916817+0000","last_fullsized":"2024-09-19T04:06:42.916817+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:24.289049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:59.737295+0000","last_change":"2024-09-19T04:04:37.647310+0000","last_active":"2024-09-19T04:06:59.737295+0000","last_peered":"2024-09-19T04:06:59.737295+0000","last_clean":"2024-09-19T04:06:59.737295+0000","last_became_active":"2024-09-19T04:04:37.646903+0000","last_became_peered":"2024-09-19T04:04:37.646903+0000","last_unstale":"2024-09-19T04:06:59.737295+0000","last_undegraded":"2024-09-19T04:06:59.737295+0000","last_fullsized":"2024-09-19T04:06:59.737295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:20.097472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909319+0000","last_change":"2024-09-19T04:04:46.375261+0000","last_active":"2024-09-19T04:06:42.909319+0000","last_peered":"2024-09-19T04:06:42.909319+0000","last_clean":"2024-09-19T04:06:42.909319+0000","last_became_active":"2024-09-19T04:04:46.374489+0000","last_became_peered":"2024-09-19T04:04:46.374489+0000","last_unstale":"2024-09-19T04:06:42.909319+0000","last_undegraded":"2024-09-19T04:06:42.909319+0000","last_fullsized":"2024-09-19T04:06:42.909319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:05.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.908952+0000","last_change":"2024-09-19T04:04:39.322265+0000","last_active":"2024-09-19T04:06:42.908952+0000","last_peered":"2024-09-19T04:06:42.908952+0000","last_clean":"2024-09-19T04:06:42.908952+0000","last_became_active":"2024-09-19T04:04:39.321938+0000","last_became_peered":"2024-09-19T04:04:39.321938+0000","last_unstale":"2024-09-19T04:06:42.908952+0000","last_undegraded":"2024-09-19T04:06:42.908952+0000","last_fullsized":"2024-09-19T04:06:42.908952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:19.220661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418797+0000","last_change":"2024-09-19T04:04:37.211330+0000","last_active":"2024-09-19T04:06:43.418797+0000","last_peered":"2024-09-19T04:06:43.418797+0000","last_clean":"2024-09-19T04:06:43.418797+0000","last_became_active":"2024-09-19T04:04:37.210788+0000","last_became_peered":"2024-09-19T04:04:37.210788+0000","last_unstale":"2024-09-19T04:06:43.418797+0000","last_undegraded":"2024-09-19T04:06:43.418797+0000","last_fullsized":"2024-09-19T04:06:43.418797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:43.169897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913106+0000","last_change":"2024-09-19T04:05:42.465969+0000","last_active":"2024-09-19T04:06:42.913106+0000","last_peered":"2024-09-19T04:06:42.913106+0000","last_clean":"2024-09-19T04:06:42.913106+0000","last_became_active":"2024-09-19T04:05:42.465621+0000","last_became_peered":"2024-09-19T04:05:42.465621+0000","last_unstale":"2024-09-19T04:06:42.913106+0000","last_undegraded":"2024-09-19T04:06:42.913106+0000","last_fullsized":"2024-09-19T04:06:42.913106+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:29.802141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,10],"acting":[5,1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294450+0000","last_change":"2024-09-19T04:04:39.329480+0000","last_active":"2024-09-19T04:06:43.294450+0000","last_peered":"2024-09-19T04:06:43.294450+0000","last_clean":"2024-09-19T04:06:43.294450+0000","last_became_active":"2024-09-19T04:04:39.329008+0000","last_became_peered":"2024-09-19T04:04:39.329008+0000","last_unstale":"2024-09-19T04:06:43.294450+0000","last_undegraded":"2024-09-19T04:06:43.294450+0000","last_fullsized":"2024-09-19T04:06:43.294450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:06.782579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915484+0000","last_change":"2024-09-19T04:04:37.216224+0000","last_active":"2024-09-19T04:06:42.915484+0000","last_peered":"2024-09-19T04:06:42.915484+0000","last_clean":"2024-09-19T04:06:42.915484+0000","last_became_active":"2024-09-19T04:04:37.215846+0000","last_became_peered":"2024-09-19T04:04:37.215846+0000","last_unstale":"2024-09-19T04:06:42.915484+0000","last_undegraded":"2024-09-19T04:06:42.915484+0000","last_fullsized":"2024-09-19T04:06:42.915484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:57.829694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418966+0000","last_change":"2024-09-19T04:03:25.869634+0000","last_active":"2024-09-19T04:06:43.418966+0000","last_peered":"2024-09-19T04:06:43.418966+0000","last_clean":"2024-09-19T04:06:43.418966+0000","last_became_active":"2024-09-19T04:03:25.869250+0000","last_became_peered":"2024-09-19T04:03:25.869250+0000","last_unstale":"2024-09-19T04:06:43.418966+0000","last_undegraded":"2024-09-19T04:06:43.418966+0000","last_fullsized":"2024-09-19T04:06:43.418966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_clean_scrub_stamp":"2024-09-19T03:59:59.253417+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:51.956932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419061+0000","last_change":"2024-09-19T04:04:46.357123+0000","last_active":"2024-09-19T04:06:43.419061+0000","last_peered":"2024-09-19T04:06:43.419061+0000","last_clean":"2024-09-19T04:06:43.419061+0000","last_became_active":"2024-09-19T04:04:46.356671+0000","last_became_peered":"2024-09-19T04:04:46.356671+0000","last_unstale":"2024-09-19T04:06:43.419061+0000","last_undegraded":"2024-09-19T04:06:43.419061+0000","last_fullsized":"2024-09-19T04:06:43.419061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:09.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419189+0000","last_change":"2024-09-19T04:04:39.322487+0000","last_active":"2024-09-19T04:06:43.419189+0000","last_peered":"2024-09-19T04:06:43.419189+0000","last_clean":"2024-09-19T04:06:43.419189+0000","last_became_active":"2024-09-19T04:04:39.321994+0000","last_became_peered":"2024-09-19T04:04:39.321994+0000","last_unstale":"2024-09-19T04:06:43.419189+0000","last_undegraded":"2024-09-19T04:06:43.419189+0000","last_fullsized":"2024-09-19T04:06:43.419189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:56.977405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":147,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:01.750953+0000","last_change":"2024-09-19T04:04:37.703452+0000","last_active":"2024-09-19T04:07:01.750953+0000","last_peered":"2024-09-19T04:07:01.750953+0000","last_clean":"2024-09-19T04:07:01.750953+0000","last_became_active":"2024-09-19T04:04:37.703048+0000","last_became_peered":"2024-09-19T04:04:37.703048+0000","last_unstale":"2024-09-19T04:07:01.750953+0000","last_undegraded":"2024-09-19T04:07:01.750953+0000","last_fullsized":"2024-09-19T04:07:01.750953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:16:29.051617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110726+0000","last_change":"2024-09-19T04:04:46.361229+0000","last_active":"2024-09-19T04:06:44.110726+0000","last_peered":"2024-09-19T04:06:44.110726+0000","last_clean":"2024-09-19T04:06:44.110726+0000","last_became_active":"2024-09-19T04:04:46.360729+0000","last_became_peered":"2024-09-19T04:04:46.360729+0000","last_unstale":"2024-09-19T04:06:44.110726+0000","last_undegraded":"2024-09-19T04:06:44.110726+0000","last_fullsized":"2024-09-19T04:06:44.110726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:48.022778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418940+0000","last_change":"2024-09-19T04:04:39.328598+0000","last_active":"2024-09-19T04:06:43.418940+0000","last_peered":"2024-09-19T04:06:43.418940+0000","last_clean":"2024-09-19T04:06:43.418940+0000","last_became_active":"2024-09-19T04:04:39.328247+0000","last_became_peered":"2024-09-19T04:04:39.328247+0000","last_unstale":"2024-09-19T04:06:43.418940+0000","last_undegraded":"2024-09-19T04:06:43.418940+0000","last_fullsized":"2024-09-19T04:06:43.418940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:34.916487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913045+0000","last_change":"2024-09-19T04:04:37.212065+0000","last_active":"2024-09-19T04:06:42.913045+0000","last_peered":"2024-09-19T04:06:42.913045+0000","last_clean":"2024-09-19T04:06:42.913045+0000","last_became_active":"2024-09-19T04:04:37.211520+0000","last_became_peered":"2024-09-19T04:04:37.211520+0000","last_unstale":"2024-09-19T04:06:42.913045+0000","last_undegraded":"2024-09-19T04:06:42.913045+0000","last_fullsized":"2024-09-19T04:06:42.913045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:53.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912491+0000","last_change":"2024-09-19T04:04:46.358558+0000","last_active":"2024-09-19T04:06:42.912491+0000","last_peered":"2024-09-19T04:06:42.912491+0000","last_clean":"2024-09-19T04:06:42.912491+0000","last_became_active":"2024-09-19T04:04:46.358158+0000","last_became_peered":"2024-09-19T04:04:46.358158+0000","last_unstale":"2024-09-19T04:06:42.912491+0000","last_undegraded":"2024-09-19T04:06:42.912491+0000","last_fullsized":"2024-09-19T04:06:42.912491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:16.627051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419310+0000","last_change":"2024-09-19T04:04:39.325187+0000","last_active":"2024-09-19T04:06:43.419310+0000","last_peered":"2024-09-19T04:06:43.419310+0000","last_clean":"2024-09-19T04:06:43.419310+0000","last_became_active":"2024-09-19T04:04:39.324644+0000","last_became_peered":"2024-09-19T04:04:39.324644+0000","last_unstale":"2024-09-19T04:06:43.419310+0000","last_undegraded":"2024-09-19T04:06:43.419310+0000","last_fullsized":"2024-09-19T04:06:43.419310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:11.879273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909362+0000","last_change":"2024-09-19T04:04:37.205389+0000","last_active":"2024-09-19T04:06:42.909362+0000","last_peered":"2024-09-19T04:06:42.909362+0000","last_clean":"2024-09-19T04:06:42.909362+0000","last_became_active":"2024-09-19T04:04:37.205117+0000","last_became_peered":"2024-09-19T04:04:37.205117+0000","last_unstale":"2024-09-19T04:06:42.909362+0000","last_undegraded":"2024-09-19T04:06:42.909362+0000","last_fullsized":"2024-09-19T04:06:42.909362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:49.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916378+0000","last_change":"2024-09-19T04:05:42.342246+0000","last_active":"2024-09-19T04:06:42.916378+0000","last_peered":"2024-09-19T04:06:42.916378+0000","last_clean":"2024-09-19T04:06:42.916378+0000","last_became_active":"2024-09-19T04:05:42.341628+0000","last_became_peered":"2024-09-19T04:05:42.341628+0000","last_unstale":"2024-09-19T04:06:42.916378+0000","last_undegraded":"2024-09-19T04:06:42.916378+0000","last_fullsized":"2024-09-19T04:06:42.916378+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:16.637647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111363+0000","last_change":"2024-09-19T04:04:39.328841+0000","last_active":"2024-09-19T04:06:44.111363+0000","last_peered":"2024-09-19T04:06:44.111363+0000","last_clean":"2024-09-19T04:06:44.111363+0000","last_became_active":"2024-09-19T04:04:39.328470+0000","last_became_peered":"2024-09-19T04:04:39.328470+0000","last_unstale":"2024-09-19T04:06:44.111363+0000","last_undegraded":"2024-09-19T04:06:44.111363+0000","last_fullsized":"2024-09-19T04:06:44.111363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:17.701469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111344+0000","last_change":"2024-09-19T04:05:42.353456+0000","last_active":"2024-09-19T04:06:44.111344+0000","last_peered":"2024-09-19T04:06:44.111344+0000","last_clean":"2024-09-19T04:06:44.111344+0000","last_became_active":"2024-09-19T04:05:42.340710+0000","last_became_peered":"2024-09-19T04:05:42.340710+0000","last_unstale":"2024-09-19T04:06:44.111344+0000","last_undegraded":"2024-09-19T04:06:44.111344+0000","last_fullsized":"2024-09-19T04:06:44.111344+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:45.262693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912021+0000","last_change":"2024-09-19T04:04:46.361199+0000","last_active":"2024-09-19T04:06:42.912021+0000","last_peered":"2024-09-19T04:06:42.912021+0000","last_clean":"2024-09-19T04:06:42.912021+0000","last_became_active":"2024-09-19T04:04:46.360639+0000","last_became_peered":"2024-09-19T04:04:46.360639+0000","last_unstale":"2024-09-19T04:06:42.912021+0000","last_undegraded":"2024-09-19T04:06:42.912021+0000","last_fullsized":"2024-09-19T04:06:42.912021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:50.185760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912115+0000","last_change":"2024-09-19T04:04:39.328529+0000","last_active":"2024-09-19T04:06:42.912115+0000","last_peered":"2024-09-19T04:06:42.912115+0000","last_clean":"2024-09-19T04:06:42.912115+0000","last_became_active":"2024-09-19T04:04:39.328174+0000","last_became_peered":"2024-09-19T04:04:39.328174+0000","last_unstale":"2024-09-19T04:06:42.912115+0000","last_undegraded":"2024-09-19T04:06:42.912115+0000","last_fullsized":"2024-09-19T04:06:42.912115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:02.503672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916702+0000","last_change":"2024-09-19T04:04:37.647919+0000","last_active":"2024-09-19T04:06:42.916702+0000","last_peered":"2024-09-19T04:06:42.916702+0000","last_clean":"2024-09-19T04:06:42.916702+0000","last_became_active":"2024-09-19T04:04:37.647305+0000","last_became_peered":"2024-09-19T04:04:37.647305+0000","last_unstale":"2024-09-19T04:06:42.916702+0000","last_undegraded":"2024-09-19T04:06:42.916702+0000","last_fullsized":"2024-09-19T04:06:42.916702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:07.035465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909285+0000","last_change":"2024-09-19T04:04:46.360362+0000","last_active":"2024-09-19T04:06:42.909285+0000","last_peered":"2024-09-19T04:06:42.909285+0000","last_clean":"2024-09-19T04:06:42.909285+0000","last_became_active":"2024-09-19T04:04:46.359470+0000","last_became_peered":"2024-09-19T04:04:46.359470+0000","last_unstale":"2024-09-19T04:06:42.909285+0000","last_undegraded":"2024-09-19T04:06:42.909285+0000","last_fullsized":"2024-09-19T04:06:42.909285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:43.804073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418669+0000","last_change":"2024-09-19T04:04:39.326090+0000","last_active":"2024-09-19T04:06:43.418669+0000","last_peered":"2024-09-19T04:06:43.418669+0000","last_clean":"2024-09-19T04:06:43.418669+0000","last_became_active":"2024-09-19T04:04:39.325751+0000","last_became_peered":"2024-09-19T04:04:39.325751+0000","last_unstale":"2024-09-19T04:06:43.418669+0000","last_undegraded":"2024-09-19T04:06:43.418669+0000","last_fullsized":"2024-09-19T04:06:43.418669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:25.357467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:59.738007+0000","last_change":"2024-09-19T04:04:37.215107+0000","last_active":"2024-09-19T04:06:59.738007+0000","last_peered":"2024-09-19T04:06:59.738007+0000","last_clean":"2024-09-19T04:06:59.738007+0000","last_became_active":"2024-09-19T04:04:37.214693+0000","last_became_peered":"2024-09-19T04:04:37.214693+0000","last_unstale":"2024-09-19T04:06:59.738007+0000","last_undegraded":"2024-09-19T04:06:59.738007+0000","last_fullsized":"2024-09-19T04:06:59.738007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:56.535316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913700+0000","last_change":"2024-09-19T04:04:46.363625+0000","last_active":"2024-09-19T04:06:42.913700+0000","last_peered":"2024-09-19T04:06:42.913700+0000","last_clean":"2024-09-19T04:06:42.913700+0000","last_became_active":"2024-09-19T04:04:46.363207+0000","last_became_peered":"2024-09-19T04:04:46.363207+0000","last_unstale":"2024-09-19T04:06:42.913700+0000","last_undegraded":"2024-09-19T04:06:42.913700+0000","last_fullsized":"2024-09-19T04:06:42.913700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:30.251168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418748+0000","last_change":"2024-09-19T04:05:42.466411+0000","last_active":"2024-09-19T04:06:43.418748+0000","last_peered":"2024-09-19T04:06:43.418748+0000","last_clean":"2024-09-19T04:06:43.418748+0000","last_became_active":"2024-09-19T04:05:42.465999+0000","last_became_peered":"2024-09-19T04:05:42.465999+0000","last_unstale":"2024-09-19T04:06:43.418748+0000","last_undegraded":"2024-09-19T04:06:43.418748+0000","last_fullsized":"2024-09-19T04:06:43.418748+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:51.289726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912476+0000","last_change":"2024-09-19T04:04:37.212103+0000","last_active":"2024-09-19T04:06:42.912476+0000","last_peered":"2024-09-19T04:06:42.912476+0000","last_clean":"2024-09-19T04:06:42.912476+0000","last_became_active":"2024-09-19T04:04:37.211739+0000","last_became_peered":"2024-09-19T04:04:37.211739+0000","last_unstale":"2024-09-19T04:06:42.912476+0000","last_undegraded":"2024-09-19T04:06:42.912476+0000","last_fullsized":"2024-09-19T04:06:42.912476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:06.588194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110915+0000","last_change":"2024-09-19T04:04:46.361975+0000","last_active":"2024-09-19T04:06:44.110915+0000","last_peered":"2024-09-19T04:06:44.110915+0000","last_clean":"2024-09-19T04:06:44.110915+0000","last_became_active":"2024-09-19T04:04:46.361492+0000","last_became_peered":"2024-09-19T04:04:46.361492+0000","last_unstale":"2024-09-19T04:06:44.110915+0000","last_undegraded":"2024-09-19T04:06:44.110915+0000","last_fullsized":"2024-09-19T04:06:44.110915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:52.723836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909144+0000","last_change":"2024-09-19T04:04:39.317756+0000","last_active":"2024-09-19T04:06:42.909144+0000","last_peered":"2024-09-19T04:06:42.909144+0000","last_clean":"2024-09-19T04:06:42.909144+0000","last_became_active":"2024-09-19T04:04:39.317322+0000","last_became_peered":"2024-09-19T04:04:39.317322+0000","last_unstale":"2024-09-19T04:06:42.909144+0000","last_undegraded":"2024-09-19T04:06:42.909144+0000","last_fullsized":"2024-09-19T04:06:42.909144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:26.291574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914459+0000","last_change":"2024-09-19T04:04:37.213892+0000","last_active":"2024-09-19T04:06:42.914459+0000","last_peered":"2024-09-19T04:06:42.914459+0000","last_clean":"2024-09-19T04:06:42.914459+0000","last_became_active":"2024-09-19T04:04:37.213350+0000","last_became_peered":"2024-09-19T04:04:37.213350+0000","last_unstale":"2024-09-19T04:06:42.914459+0000","last_undegraded":"2024-09-19T04:06:42.914459+0000","last_fullsized":"2024-09-19T04:06:42.914459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:40.065717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912727+0000","last_change":"2024-09-19T04:04:46.360566+0000","last_active":"2024-09-19T04:06:42.912727+0000","last_peered":"2024-09-19T04:06:42.912727+0000","last_clean":"2024-09-19T04:06:42.912727+0000","last_became_active":"2024-09-19T04:04:46.360143+0000","last_became_peered":"2024-09-19T04:04:46.360143+0000","last_unstale":"2024-09-19T04:06:42.912727+0000","last_undegraded":"2024-09-19T04:06:42.912727+0000","last_fullsized":"2024-09-19T04:06:42.912727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:59.997088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916894+0000","last_change":"2024-09-19T04:04:39.318363+0000","last_active":"2024-09-19T04:06:42.916894+0000","last_peered":"2024-09-19T04:06:42.916894+0000","last_clean":"2024-09-19T04:06:42.916894+0000","last_became_active":"2024-09-19T04:04:39.318003+0000","last_became_peered":"2024-09-19T04:04:39.318003+0000","last_unstale":"2024-09-19T04:06:42.916894+0000","last_undegraded":"2024-09-19T04:06:42.916894+0000","last_fullsized":"2024-09-19T04:06:42.916894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:54.128795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:02.773578+0000","last_change":"2024-09-19T04:05:42.349944+0000","last_active":"2024-09-19T04:07:02.773578+0000","last_peered":"2024-09-19T04:07:02.773578+0000","last_clean":"2024-09-19T04:07:02.773578+0000","last_became_active":"2024-09-19T04:05:42.339605+0000","last_became_peered":"2024-09-19T04:05:42.339605+0000","last_unstale":"2024-09-19T04:07:02.773578+0000","last_undegraded":"2024-09-19T04:07:02.773578+0000","last_fullsized":"2024-09-19T04:07:02.773578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:17.508316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62633,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":168,"num_write":41,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":60938,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912879+0000","last_change":"2024-09-19T04:04:46.362967+0000","last_active":"2024-09-19T04:06:42.912879+0000","last_peered":"2024-09-19T04:06:42.912879+0000","last_clean":"2024-09-19T04:06:42.912879+0000","last_became_active":"2024-09-19T04:04:46.362529+0000","last_became_peered":"2024-09-19T04:04:46.362529+0000","last_unstale":"2024-09-19T04:06:42.912879+0000","last_undegraded":"2024-09-19T04:06:42.912879+0000","last_fullsized":"2024-09-19T04:06:42.912879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:01.821697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110944+0000","last_change":"2024-09-19T04:04:39.323499+0000","last_active":"2024-09-19T04:06:44.110944+0000","last_peered":"2024-09-19T04:06:44.110944+0000","last_clean":"2024-09-19T04:06:44.110944+0000","last_became_active":"2024-09-19T04:04:39.323076+0000","last_became_peered":"2024-09-19T04:04:39.323076+0000","last_unstale":"2024-09-19T04:06:44.110944+0000","last_undegraded":"2024-09-19T04:06:44.110944+0000","last_fullsized":"2024-09-19T04:06:44.110944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:10.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912875+0000","last_change":"2024-09-19T04:04:37.647927+0000","last_active":"2024-09-19T04:06:42.912875+0000","last_peered":"2024-09-19T04:06:42.912875+0000","last_clean":"2024-09-19T04:06:42.912875+0000","last_became_active":"2024-09-19T04:04:37.647279+0000","last_became_peered":"2024-09-19T04:04:37.647279+0000","last_unstale":"2024-09-19T04:06:42.912875+0000","last_undegraded":"2024-09-19T04:06:42.912875+0000","last_fullsized":"2024-09-19T04:06:42.912875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:05:50.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914938+0000","last_change":"2024-09-19T04:04:46.363562+0000","last_active":"2024-09-19T04:06:42.914938+0000","last_peered":"2024-09-19T04:06:42.914938+0000","last_clean":"2024-09-19T04:06:42.914938+0000","last_became_active":"2024-09-19T04:04:46.362915+0000","last_became_peered":"2024-09-19T04:04:46.362915+0000","last_unstale":"2024-09-19T04:06:42.914938+0000","last_undegraded":"2024-09-19T04:06:42.914938+0000","last_fullsized":"2024-09-19T04:06:42.914938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:08.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.420917+0000","last_change":"2024-09-19T04:06:43.420595+0000","last_active":"2024-09-19T04:06:43.420917+0000","last_peered":"2024-09-19T04:06:43.420917+0000","last_clean":"2024-09-19T04:06:43.420917+0000","last_became_active":"2024-09-19T04:06:43.420316+0000","last_became_peered":"2024-09-19T04:06:43.420316+0000","last_unstale":"2024-09-19T04:06:43.420917+0000","last_undegraded":"2024-09-19T04:06:43.420917+0000","last_fullsized":"2024-09-19T04:06:43.420917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:12:54.798028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419021+0000","last_change":"2024-09-19T04:04:37.212393+0000","last_active":"2024-09-19T04:06:43.419021+0000","last_peered":"2024-09-19T04:06:43.419021+0000","last_clean":"2024-09-19T04:06:43.419021+0000","last_became_active":"2024-09-19T04:04:37.212013+0000","last_became_peered":"2024-09-19T04:04:37.212013+0000","last_unstale":"2024-09-19T04:06:43.419021+0000","last_undegraded":"2024-09-19T04:06:43.419021+0000","last_fullsized":"2024-09-19T04:06:43.419021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:37.981075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916368+0000","last_change":"2024-09-19T04:05:43.151378+0000","last_active":"2024-09-19T04:06:42.916368+0000","last_peered":"2024-09-19T04:06:42.916368+0000","last_clean":"2024-09-19T04:06:42.916368+0000","last_became_active":"2024-09-19T04:05:43.150907+0000","last_became_peered":"2024-09-19T04:05:43.150907+0000","last_unstale":"2024-09-19T04:06:42.916368+0000","last_undegraded":"2024-09-19T04:06:42.916368+0000","last_fullsized":"2024-09-19T04:06:42.916368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:09.542371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111185+0000","last_change":"2024-09-19T04:04:39.319705+0000","last_active":"2024-09-19T04:06:44.111185+0000","last_peered":"2024-09-19T04:06:44.111185+0000","last_clean":"2024-09-19T04:06:44.111185+0000","last_became_active":"2024-09-19T04:04:39.318065+0000","last_became_peered":"2024-09-19T04:04:39.318065+0000","last_unstale":"2024-09-19T04:06:44.111185+0000","last_undegraded":"2024-09-19T04:06:44.111185+0000","last_fullsized":"2024-09-19T04:06:44.111185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:15.630927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":246,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:02.770896+0000","last_change":"2024-09-19T04:04:37.215192+0000","last_active":"2024-09-19T04:07:02.770896+0000","last_peered":"2024-09-19T04:07:02.770896+0000","last_clean":"2024-09-19T04:07:02.770896+0000","last_became_active":"2024-09-19T04:04:37.214872+0000","last_became_peered":"2024-09-19T04:04:37.214872+0000","last_unstale":"2024-09-19T04:07:02.770896+0000","last_undegraded":"2024-09-19T04:07:02.770896+0000","last_fullsized":"2024-09-19T04:07:02.770896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:18.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294101+0000","last_change":"2024-09-19T04:04:46.358113+0000","last_active":"2024-09-19T04:06:43.294101+0000","last_peered":"2024-09-19T04:06:43.294101+0000","last_clean":"2024-09-19T04:06:43.294101+0000","last_became_active":"2024-09-19T04:04:46.356604+0000","last_became_peered":"2024-09-19T04:04:46.356604+0000","last_unstale":"2024-09-19T04:06:43.294101+0000","last_undegraded":"2024-09-19T04:06:43.294101+0000","last_fullsized":"2024-09-19T04:06:43.294101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:38.147738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912583+0000","last_change":"2024-09-19T04:04:46.354516+0000","last_active":"2024-09-19T04:06:42.912583+0000","last_peered":"2024-09-19T04:06:42.912583+0000","last_clean":"2024-09-19T04:06:42.912583+0000","last_became_active":"2024-09-19T04:04:46.354034+0000","last_became_peered":"2024-09-19T04:04:46.354034+0000","last_unstale":"2024-09-19T04:06:42.912583+0000","last_undegraded":"2024-09-19T04:06:42.912583+0000","last_fullsized":"2024-09-19T04:06:42.912583+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:21.946746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912869+0000","last_change":"2024-09-19T04:04:39.311324+0000","last_active":"2024-09-19T04:06:42.912869+0000","last_peered":"2024-09-19T04:06:42.912869+0000","last_clean":"2024-09-19T04:06:42.912869+0000","last_became_active":"2024-09-19T04:04:39.310977+0000","last_became_peered":"2024-09-19T04:04:39.310977+0000","last_unstale":"2024-09-19T04:06:42.912869+0000","last_undegraded":"2024-09-19T04:06:42.912869+0000","last_fullsized":"2024-09-19T04:06:42.912869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:43.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912261+0000","last_change":"2024-09-19T04:04:37.212773+0000","last_active":"2024-09-19T04:06:42.912261+0000","last_peered":"2024-09-19T04:06:42.912261+0000","last_clean":"2024-09-19T04:06:42.912261+0000","last_became_active":"2024-09-19T04:04:37.212405+0000","last_became_peered":"2024-09-19T04:04:37.212405+0000","last_unstale":"2024-09-19T04:06:42.912261+0000","last_undegraded":"2024-09-19T04:06:42.912261+0000","last_fullsized":"2024-09-19T04:06:42.912261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:29.020122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111297+0000","last_change":"2024-09-19T04:04:46.363052+0000","last_active":"2024-09-19T04:06:44.111297+0000","last_peered":"2024-09-19T04:06:44.111297+0000","last_clean":"2024-09-19T04:06:44.111297+0000","last_became_active":"2024-09-19T04:04:46.362588+0000","last_became_peered":"2024-09-19T04:04:46.362588+0000","last_unstale":"2024-09-19T04:06:44.111297+0000","last_undegraded":"2024-09-19T04:06:44.111297+0000","last_fullsized":"2024-09-19T04:06:44.111297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:52.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294114+0000","last_change":"2024-09-19T04:04:39.313686+0000","last_active":"2024-09-19T04:06:43.294114+0000","last_peered":"2024-09-19T04:06:43.294114+0000","last_clean":"2024-09-19T04:06:43.294114+0000","last_became_active":"2024-09-19T04:04:39.313386+0000","last_became_peered":"2024-09-19T04:04:39.313386+0000","last_unstale":"2024-09-19T04:06:43.294114+0000","last_undegraded":"2024-09-19T04:06:43.294114+0000","last_fullsized":"2024-09-19T04:06:43.294114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:08.274930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.908889+0000","last_change":"2024-09-19T04:04:37.207926+0000","last_active":"2024-09-19T04:06:42.908889+0000","last_peered":"2024-09-19T04:06:42.908889+0000","last_clean":"2024-09-19T04:06:42.908889+0000","last_became_active":"2024-09-19T04:04:37.207538+0000","last_became_peered":"2024-09-19T04:04:37.207538+0000","last_unstale":"2024-09-19T04:06:42.908889+0000","last_undegraded":"2024-09-19T04:06:42.908889+0000","last_fullsized":"2024-09-19T04:06:42.908889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:29.621638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913935+0000","last_change":"2024-09-19T04:04:37.212829+0000","last_active":"2024-09-19T04:06:42.913935+0000","last_peered":"2024-09-19T04:06:42.913935+0000","last_clean":"2024-09-19T04:06:42.913935+0000","last_became_active":"2024-09-19T04:04:37.212294+0000","last_became_peered":"2024-09-19T04:04:37.212294+0000","last_unstale":"2024-09-19T04:06:42.913935+0000","last_undegraded":"2024-09-19T04:06:42.913935+0000","last_fullsized":"2024-09-19T04:06:42.913935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:34.479708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912531+0000","last_change":"2024-09-19T04:04:39.328622+0000","last_active":"2024-09-19T04:06:42.912531+0000","last_peered":"2024-09-19T04:06:42.912531+0000","last_clean":"2024-09-19T04:06:42.912531+0000","last_became_active":"2024-09-19T04:04:39.328290+0000","last_became_peered":"2024-09-19T04:04:39.328290+0000","last_unstale":"2024-09-19T04:06:42.912531+0000","last_undegraded":"2024-09-19T04:06:42.912531+0000","last_fullsized":"2024-09-19T04:06:42.912531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:04.290767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914155+0000","last_change":"2024-09-19T04:04:46.362853+0000","last_active":"2024-09-19T04:06:42.914155+0000","last_peered":"2024-09-19T04:06:42.914155+0000","last_clean":"2024-09-19T04:06:42.914155+0000","last_became_active":"2024-09-19T04:04:46.362392+0000","last_became_peered":"2024-09-19T04:04:46.362392+0000","last_unstale":"2024-09-19T04:06:42.914155+0000","last_undegraded":"2024-09-19T04:06:42.914155+0000","last_fullsized":"2024-09-19T04:06:42.914155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:40.016502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":142,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:57.744434+0000","last_change":"2024-09-19T04:04:37.213248+0000","last_active":"2024-09-19T04:06:57.744434+0000","last_peered":"2024-09-19T04:06:57.744434+0000","last_clean":"2024-09-19T04:06:57.744434+0000","last_became_active":"2024-09-19T04:04:37.212639+0000","last_became_peered":"2024-09-19T04:04:37.212639+0000","last_unstale":"2024-09-19T04:06:57.744434+0000","last_undegraded":"2024-09-19T04:06:57.744434+0000","last_fullsized":"2024-09-19T04:06:57.744434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:45:05.917792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916668+0000","last_change":"2024-09-19T04:04:39.330160+0000","last_active":"2024-09-19T04:06:42.916668+0000","last_peered":"2024-09-19T04:06:42.916668+0000","last_clean":"2024-09-19T04:06:42.916668+0000","last_became_active":"2024-09-19T04:04:39.329828+0000","last_became_peered":"2024-09-19T04:04:39.329828+0000","last_unstale":"2024-09-19T04:06:42.916668+0000","last_undegraded":"2024-09-19T04:06:42.916668+0000","last_fullsized":"2024-09-19T04:06:42.916668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:57.712550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914137+0000","last_change":"2024-09-19T04:04:46.360459+0000","last_active":"2024-09-19T04:06:42.914137+0000","last_peered":"2024-09-19T04:06:42.914137+0000","last_clean":"2024-09-19T04:06:42.914137+0000","last_became_active":"2024-09-19T04:04:46.359914+0000","last_became_peered":"2024-09-19T04:04:46.359914+0000","last_unstale":"2024-09-19T04:06:42.914137+0000","last_undegraded":"2024-09-19T04:06:42.914137+0000","last_fullsized":"2024-09-19T04:06:42.914137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:30.380358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110927+0000","last_change":"2024-09-19T04:04:39.324847+0000","last_active":"2024-09-19T04:06:44.110927+0000","last_peered":"2024-09-19T04:06:44.110927+0000","last_clean":"2024-09-19T04:06:44.110927+0000","last_became_active":"2024-09-19T04:04:39.324402+0000","last_became_peered":"2024-09-19T04:04:39.324402+0000","last_unstale":"2024-09-19T04:06:44.110927+0000","last_undegraded":"2024-09-19T04:06:44.110927+0000","last_fullsized":"2024-09-19T04:06:44.110927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:41.996135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.918796+0000","last_change":"2024-09-19T04:06:42.918320+0000","last_active":"2024-09-19T04:06:42.918796+0000","last_peered":"2024-09-19T04:06:42.918796+0000","last_clean":"2024-09-19T04:06:42.918796+0000","last_became_active":"2024-09-19T04:06:42.917819+0000","last_became_peered":"2024-09-19T04:06:42.917819+0000","last_unstale":"2024-09-19T04:06:42.918796+0000","last_undegraded":"2024-09-19T04:06:42.918796+0000","last_fullsized":"2024-09-19T04:06:42.918796+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:02.885774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,8],"acting":[2,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914539+0000","last_change":"2024-09-19T04:04:39.318556+0000","last_active":"2024-09-19T04:06:42.914539+0000","last_peered":"2024-09-19T04:06:42.914539+0000","last_clean":"2024-09-19T04:06:42.914539+0000","last_became_active":"2024-09-19T04:04:39.318200+0000","last_became_peered":"2024-09-19T04:04:39.318200+0000","last_unstale":"2024-09-19T04:06:42.914539+0000","last_undegraded":"2024-09-19T04:06:42.914539+0000","last_fullsized":"2024-09-19T04:06:42.914539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:40.847440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912503+0000","last_change":"2024-09-19T04:04:39.320698+0000","last_active":"2024-09-19T04:06:42.912503+0000","last_peered":"2024-09-19T04:06:42.912503+0000","last_clean":"2024-09-19T04:06:42.912503+0000","last_became_active":"2024-09-19T04:04:39.320419+0000","last_became_peered":"2024-09-19T04:04:39.320419+0000","last_unstale":"2024-09-19T04:06:42.912503+0000","last_undegraded":"2024-09-19T04:06:42.912503+0000","last_fullsized":"2024-09-19T04:06:42.912503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:32.708394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909421+0000","last_change":"2024-09-19T04:04:39.319886+0000","last_active":"2024-09-19T04:06:42.909421+0000","last_peered":"2024-09-19T04:06:42.909421+0000","last_clean":"2024-09-19T04:06:42.909421+0000","last_became_active":"2024-09-19T04:04:39.319217+0000","last_became_peered":"2024-09-19T04:04:39.319217+0000","last_unstale":"2024-09-19T04:06:42.909421+0000","last_undegraded":"2024-09-19T04:06:42.909421+0000","last_fullsized":"2024-09-19T04:06:42.909421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:38.829608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111374+0000","last_change":"2024-09-19T04:04:39.327639+0000","last_active":"2024-09-19T04:06:44.111374+0000","last_peered":"2024-09-19T04:06:44.111374+0000","last_clean":"2024-09-19T04:06:44.111374+0000","last_became_active":"2024-09-19T04:04:39.327349+0000","last_became_peered":"2024-09-19T04:04:39.327349+0000","last_unstale":"2024-09-19T04:06:44.111374+0000","last_undegraded":"2024-09-19T04:06:44.111374+0000","last_fullsized":"2024-09-19T04:06:44.111374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:31.056609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914540+0000","last_change":"2024-09-19T04:04:39.320253+0000","last_active":"2024-09-19T04:06:42.914540+0000","last_peered":"2024-09-19T04:06:42.914540+0000","last_clean":"2024-09-19T04:06:42.914540+0000","last_became_active":"2024-09-19T04:04:39.319851+0000","last_became_peered":"2024-09-19T04:04:39.319851+0000","last_unstale":"2024-09-19T04:06:42.914540+0000","last_undegraded":"2024-09-19T04:06:42.914540+0000","last_fullsized":"2024-09-19T04:06:42.914540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:53.631216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294843+0000","last_change":"2024-09-19T04:04:39.322779+0000","last_active":"2024-09-19T04:06:43.294843+0000","last_peered":"2024-09-19T04:06:43.294843+0000","last_clean":"2024-09-19T04:06:43.294843+0000","last_became_active":"2024-09-19T04:04:39.322498+0000","last_became_peered":"2024-09-19T04:04:39.322498+0000","last_unstale":"2024-09-19T04:06:43.294843+0000","last_undegraded":"2024-09-19T04:06:43.294843+0000","last_fullsized":"2024-09-19T04:06:43.294843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:24:11.956002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.911989+0000","last_change":"2024-09-19T04:04:39.322759+0000","last_active":"2024-09-19T04:06:42.911989+0000","last_peered":"2024-09-19T04:06:42.911989+0000","last_clean":"2024-09-19T04:06:42.911989+0000","last_became_active":"2024-09-19T04:04:39.322429+0000","last_became_peered":"2024-09-19T04:04:39.322429+0000","last_unstale":"2024-09-19T04:06:42.911989+0000","last_undegraded":"2024-09-19T04:06:42.911989+0000","last_fullsized":"2024-09-19T04:06:42.911989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:36.069931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294608+0000","last_change":"2024-09-19T04:04:39.320078+0000","last_active":"2024-09-19T04:06:43.294608+0000","last_peered":"2024-09-19T04:06:43.294608+0000","last_clean":"2024-09-19T04:06:43.294608+0000","last_became_active":"2024-09-19T04:04:39.319755+0000","last_became_peered":"2024-09-19T04:04:39.319755+0000","last_unstale":"2024-09-19T04:06:43.294608+0000","last_undegraded":"2024-09-19T04:06:43.294608+0000","last_fullsized":"2024-09-19T04:06:43.294608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:07.651712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111342+0000","last_change":"2024-09-19T04:04:39.319839+0000","last_active":"2024-09-19T04:06:44.111342+0000","last_peered":"2024-09-19T04:06:44.111342+0000","last_clean":"2024-09-19T04:06:44.111342+0000","last_became_active":"2024-09-19T04:04:39.319527+0000","last_became_peered":"2024-09-19T04:04:39.319527+0000","last_unstale":"2024-09-19T04:06:44.111342+0000","last_undegraded":"2024-09-19T04:06:44.111342+0000","last_fullsized":"2024-09-19T04:06:44.111342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:31.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909471+0000","last_change":"2024-09-19T04:04:39.315975+0000","last_active":"2024-09-19T04:06:42.909471+0000","last_peered":"2024-09-19T04:06:42.909471+0000","last_clean":"2024-09-19T04:06:42.909471+0000","last_became_active":"2024-09-19T04:04:39.315671+0000","last_became_peered":"2024-09-19T04:04:39.315671+0000","last_unstale":"2024-09-19T04:06:42.909471+0000","last_undegraded":"2024-09-19T04:06:42.909471+0000","last_fullsized":"2024-09-19T04:06:42.909471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:16.349853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111082+0000","last_change":"2024-09-19T04:04:39.316278+0000","last_active":"2024-09-19T04:06:44.111082+0000","last_peered":"2024-09-19T04:06:44.111082+0000","last_clean":"2024-09-19T04:06:44.111082+0000","last_became_active":"2024-09-19T04:04:39.315876+0000","last_became_peered":"2024-09-19T04:04:39.315876+0000","last_unstale":"2024-09-19T04:06:44.111082+0000","last_undegraded":"2024-09-19T04:06:44.111082+0000","last_fullsized":"2024-09-19T04:06:44.111082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:23.544405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914652+0000","last_change":"2024-09-19T04:04:39.318952+0000","last_active":"2024-09-19T04:06:42.914652+0000","last_peered":"2024-09-19T04:06:42.914652+0000","last_clean":"2024-09-19T04:06:42.914652+0000","last_became_active":"2024-09-19T04:04:39.318598+0000","last_became_peered":"2024-09-19T04:04:39.318598+0000","last_unstale":"2024-09-19T04:06:42.914652+0000","last_undegraded":"2024-09-19T04:06:42.914652+0000","last_fullsized":"2024-09-19T04:06:42.914652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:12.654611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913733+0000","last_change":"2024-09-19T04:04:39.311334+0000","last_active":"2024-09-19T04:06:42.913733+0000","last_peered":"2024-09-19T04:06:42.913733+0000","last_clean":"2024-09-19T04:06:42.913733+0000","last_became_active":"2024-09-19T04:04:39.311038+0000","last_became_peered":"2024-09-19T04:04:39.311038+0000","last_unstale":"2024-09-19T04:06:42.913733+0000","last_undegraded":"2024-09-19T04:06:42.913733+0000","last_fullsized":"2024-09-19T04:06:42.913733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:41.655363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111491+0000","last_change":"2024-09-19T04:04:39.319529+0000","last_active":"2024-09-19T04:06:44.111491+0000","last_peered":"2024-09-19T04:06:44.111491+0000","last_clean":"2024-09-19T04:06:44.111491+0000","last_became_active":"2024-09-19T04:04:39.318787+0000","last_became_peered":"2024-09-19T04:04:39.318787+0000","last_unstale":"2024-09-19T04:06:44.111491+0000","last_undegraded":"2024-09-19T04:06:44.111491+0000","last_fullsized":"2024-09-19T04:06:44.111491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.209422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110641+0000","last_change":"2024-09-19T04:04:39.320320+0000","last_active":"2024-09-19T04:06:44.110641+0000","last_peered":"2024-09-19T04:06:44.110641+0000","last_clean":"2024-09-19T04:06:44.110641+0000","last_became_active":"2024-09-19T04:04:39.319091+0000","last_became_peered":"2024-09-19T04:04:39.319091+0000","last_unstale":"2024-09-19T04:06:44.110641+0000","last_undegraded":"2024-09-19T04:06:44.110641+0000","last_fullsized":"2024-09-19T04:06:44.110641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:39.495079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912247+0000","last_change":"2024-09-19T04:04:39.323270+0000","last_active":"2024-09-19T04:06:42.912247+0000","last_peered":"2024-09-19T04:06:42.912247+0000","last_clean":"2024-09-19T04:06:42.912247+0000","last_became_active":"2024-09-19T04:04:39.322890+0000","last_became_peered":"2024-09-19T04:04:39.322890+0000","last_unstale":"2024-09-19T04:06:42.912247+0000","last_undegraded":"2024-09-19T04:06:42.912247+0000","last_fullsized":"2024-09-19T04:06:42.912247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:07.311654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912217+0000","last_change":"2024-09-19T04:04:39.317098+0000","last_active":"2024-09-19T04:06:42.912217+0000","last_peered":"2024-09-19T04:06:42.912217+0000","last_clean":"2024-09-19T04:06:42.912217+0000","last_became_active":"2024-09-19T04:04:39.316792+0000","last_became_peered":"2024-09-19T04:04:39.316792+0000","last_unstale":"2024-09-19T04:06:42.912217+0000","last_undegraded":"2024-09-19T04:06:42.912217+0000","last_fullsized":"2024-09-19T04:06:42.912217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.571149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909342+0000","last_change":"2024-09-19T04:04:39.326473+0000","last_active":"2024-09-19T04:06:42.909342+0000","last_peered":"2024-09-19T04:06:42.909342+0000","last_clean":"2024-09-19T04:06:42.909342+0000","last_became_active":"2024-09-19T04:04:39.325506+0000","last_became_peered":"2024-09-19T04:04:39.325506+0000","last_unstale":"2024-09-19T04:06:42.909342+0000","last_undegraded":"2024-09-19T04:06:42.909342+0000","last_fullsized":"2024-09-19T04:06:42.909342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:40.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915197+0000","last_change":"2024-09-19T04:04:39.323536+0000","last_active":"2024-09-19T04:06:42.915197+0000","last_peered":"2024-09-19T04:06:42.915197+0000","last_clean":"2024-09-19T04:06:42.915197+0000","last_became_active":"2024-09-19T04:04:39.323224+0000","last_became_peered":"2024-09-19T04:04:39.323224+0000","last_unstale":"2024-09-19T04:06:42.915197+0000","last_undegraded":"2024-09-19T04:06:42.915197+0000","last_fullsized":"2024-09-19T04:06:42.915197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:54.640599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419102+0000","last_change":"2024-09-19T04:04:39.323802+0000","last_active":"2024-09-19T04:06:43.419102+0000","last_peered":"2024-09-19T04:06:43.419102+0000","last_clean":"2024-09-19T04:06:43.419102+0000","last_became_active":"2024-09-19T04:04:39.323031+0000","last_became_peered":"2024-09-19T04:04:39.323031+0000","last_unstale":"2024-09-19T04:06:43.419102+0000","last_undegraded":"2024-09-19T04:06:43.419102+0000","last_fullsized":"2024-09-19T04:06:43.419102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:12.467182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914624+0000","last_change":"2024-09-19T04:04:39.324870+0000","last_active":"2024-09-19T04:06:42.914624+0000","last_peered":"2024-09-19T04:06:42.914624+0000","last_clean":"2024-09-19T04:06:42.914624+0000","last_became_active":"2024-09-19T04:04:39.324551+0000","last_became_peered":"2024-09-19T04:04:39.324551+0000","last_unstale":"2024-09-19T04:06:42.914624+0000","last_undegraded":"2024-09-19T04:06:42.914624+0000","last_fullsized":"2024-09-19T04:06:42.914624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:34.766437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111230+0000","last_change":"2024-09-19T04:04:39.329086+0000","last_active":"2024-09-19T04:06:44.111230+0000","last_peered":"2024-09-19T04:06:44.111230+0000","last_clean":"2024-09-19T04:06:44.111230+0000","last_became_active":"2024-09-19T04:04:39.328651+0000","last_became_peered":"2024-09-19T04:04:39.328651+0000","last_unstale":"2024-09-19T04:06:44.111230+0000","last_undegraded":"2024-09-19T04:06:44.111230+0000","last_fullsized":"2024-09-19T04:06:44.111230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:12.079162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294060+0000","last_change":"2024-09-19T04:04:39.325427+0000","last_active":"2024-09-19T04:06:43.294060+0000","last_peered":"2024-09-19T04:06:43.294060+0000","last_clean":"2024-09-19T04:06:43.294060+0000","last_became_active":"2024-09-19T04:04:39.325113+0000","last_became_peered":"2024-09-19T04:04:39.325113+0000","last_unstale":"2024-09-19T04:06:43.294060+0000","last_undegraded":"2024-09-19T04:06:43.294060+0000","last_fullsized":"2024-09-19T04:06:43.294060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:15.844503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418560+0000","last_change":"2024-09-19T04:04:39.325032+0000","last_active":"2024-09-19T04:06:43.418560+0000","last_peered":"2024-09-19T04:06:43.418560+0000","last_clean":"2024-09-19T04:06:43.418560+0000","last_became_active":"2024-09-19T04:04:39.324618+0000","last_became_peered":"2024-09-19T04:04:39.324618+0000","last_unstale":"2024-09-19T04:06:43.418560+0000","last_undegraded":"2024-09-19T04:06:43.418560+0000","last_fullsized":"2024-09-19T04:06:43.418560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:58.748613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913935+0000","last_change":"2024-09-19T04:04:39.318336+0000","last_active":"2024-09-19T04:06:42.913935+0000","last_peered":"2024-09-19T04:06:42.913935+0000","last_clean":"2024-09-19T04:06:42.913935+0000","last_became_active":"2024-09-19T04:04:39.317032+0000","last_became_peered":"2024-09-19T04:04:39.317032+0000","last_unstale":"2024-09-19T04:06:42.913935+0000","last_undegraded":"2024-09-19T04:06:42.913935+0000","last_fullsized":"2024-09-19T04:06:42.913935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:04.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916560+0000","last_change":"2024-09-19T04:04:39.321909+0000","last_active":"2024-09-19T04:06:42.916560+0000","last_peered":"2024-09-19T04:06:42.916560+0000","last_clean":"2024-09-19T04:06:42.916560+0000","last_became_active":"2024-09-19T04:04:39.320522+0000","last_became_peered":"2024-09-19T04:04:39.320522+0000","last_unstale":"2024-09-19T04:06:42.916560+0000","last_undegraded":"2024-09-19T04:06:42.916560+0000","last_fullsized":"2024-09-19T04:06:42.916560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:57.270998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915032+0000","last_change":"2024-09-19T04:04:39.324610+0000","last_active":"2024-09-19T04:06:42.915032+0000","last_peered":"2024-09-19T04:06:42.915032+0000","last_clean":"2024-09-19T04:06:42.915032+0000","last_became_active":"2024-09-19T04:04:39.324190+0000","last_became_peered":"2024-09-19T04:04:39.324190+0000","last_unstale":"2024-09-19T04:06:42.915032+0000","last_undegraded":"2024-09-19T04:06:42.915032+0000","last_fullsized":"2024-09-19T04:06:42.915032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:51.418210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912621+0000","last_change":"2024-09-19T04:04:39.324220+0000","last_active":"2024-09-19T04:06:42.912621+0000","last_peered":"2024-09-19T04:06:42.912621+0000","last_clean":"2024-09-19T04:06:42.912621+0000","last_became_active":"2024-09-19T04:04:39.323890+0000","last_became_peered":"2024-09-19T04:04:39.323890+0000","last_unstale":"2024-09-19T04:06:42.912621+0000","last_undegraded":"2024-09-19T04:06:42.912621+0000","last_fullsized":"2024-09-19T04:06:42.912621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:15.475088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912692+0000","last_change":"2024-09-19T04:04:39.317240+0000","last_active":"2024-09-19T04:06:42.912692+0000","last_peered":"2024-09-19T04:06:42.912692+0000","last_clean":"2024-09-19T04:06:42.912692+0000","last_became_active":"2024-09-19T04:04:39.316740+0000","last_became_peered":"2024-09-19T04:04:39.316740+0000","last_unstale":"2024-09-19T04:06:42.912692+0000","last_undegraded":"2024-09-19T04:06:42.912692+0000","last_fullsized":"2024-09-19T04:06:42.912692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:46.705467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111449+0000","last_change":"2024-09-19T04:04:39.319665+0000","last_active":"2024-09-19T04:06:44.111449+0000","last_peered":"2024-09-19T04:06:44.111449+0000","last_clean":"2024-09-19T04:06:44.111449+0000","last_became_active":"2024-09-19T04:04:39.319227+0000","last_became_peered":"2024-09-19T04:04:39.319227+0000","last_unstale":"2024-09-19T04:06:44.111449+0000","last_undegraded":"2024-09-19T04:06:44.111449+0000","last_fullsized":"2024-09-19T04:06:44.111449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:45.581254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69589,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1035,"num_read_kb":599,"num_write":256,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":60960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":208767,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248101,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481834,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1036288,"data_stored":529087,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710767,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1183744,"data_stored":1178206,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874293,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":533091,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":529351,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1105920,"data_stored":1126013,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":525033,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1114112,"data_stored":1121061,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1069056,"data_stored":590898,"data_compressed":39195,"data_compressed_allocated":53248,"data_compressed_original":139264,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":524207,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":525749,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1101824,"data_stored":588388,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":525955,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":63025,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:07:06.115 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:07:06.375 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:06.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:05 smithi111 ceph-mon[35365]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:07:06.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:05 smithi111 ceph-mon[35365]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:07:06.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:05 smithi111 ceph-mon[35365]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:06.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:05 smithi019 ceph-mon[30938]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:07:06.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:05 smithi019 ceph-mon[30938]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:07:06.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:05 smithi019 ceph-mon[30938]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:06.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:05 smithi196 ceph-mon[35731]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T04:07:06.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:05 smithi196 ceph-mon[35731]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T04:07:06.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:05 smithi196 ceph-mon[35731]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:06.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 -- 172.21.15.19:0/1021778094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528106f10 msgr2=0x7f852810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:06.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 --2- 172.21.15.19:0/1021778094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528106f10 0x7f852810ea90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8520009ca0 tx=0x7f852002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:07:06.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 -- 172.21.15.19:0/1021778094 shutdown_connections 2024-09-19T04:07:06.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 --2- 172.21.15.19:0/1021778094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528106f10 0x7f852810ea90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 --2- 172.21.15.19:0/1021778094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8528102790 0x7f85281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 --2- 172.21.15.19:0/1021778094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8528101de0 0x7f85281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 -- 172.21.15.19:0/1021778094 >> 172.21.15.19:0/1021778094 conn(0x7f85280fd8c0 msgr2=0x7f85280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 -- 172.21.15.19:0/1021778094 shutdown_connections 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.857+0000 7f852e7c3640 1 -- 172.21.15.19:0/1021778094 wait complete. 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f852e7c3640 1 Processor -- start 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f852e7c3640 1 -- start start 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f852e7c3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8528101de0 0x7f8528198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f852e7c3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528102790 0x7f8528198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f852e7c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8528106f10 0x7f852819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f852e7c3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8528111f70 con 0x7f8528102790 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f852e7c3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8528111df0 con 0x7f8528101de0 2024-09-19T04:07:06.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f852e7c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85281120f0 con 0x7f8528106f10 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.858+0000 7f85277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528102790 0x7f8528198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528102790 0x7f8528198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38956/0 (socket says 172.21.15.19:38956) 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85277fe640 1 -- 172.21.15.19:0/3568862232 learned_addr learned my addr 172.21.15.19:0/3568862232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f8527fff640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8528101de0 0x7f8528198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f852cd60640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8528106f10 0x7f852819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85277fe640 1 -- 172.21.15.19:0/3568862232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8528106f10 msgr2=0x7f852819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85277fe640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8528106f10 0x7f852819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85277fe640 1 -- 172.21.15.19:0/3568862232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8528101de0 msgr2=0x7f8528198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85277fe640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8528101de0 0x7f8528198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85277fe640 1 -- 172.21.15.19:0/3568862232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852819d2f0 con 0x7f8528102790 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f852cd60640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8528106f10 0x7f852819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85277fe640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528102790 0x7f8528198880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c920 tx=0x7f851c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85257fa640 1 -- 172.21.15.19:0/3568862232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c007c70 con 0x7f8528102790 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85257fa640 1 -- 172.21.15.19:0/3568862232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f851c007e10 con 0x7f8528102790 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852819d5e0 con 0x7f8528102790 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f85257fa640 1 -- 172.21.15.19:0/3568862232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c018890 con 0x7f8528102790 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.859+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85281a4ec0 con 0x7f8528102790 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.860+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85281021c0 con 0x7f8528102790 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.862+0000 7f85257fa640 1 -- 172.21.15.19:0/3568862232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f851c018a70 con 0x7f8528102790 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.862+0000 7f85257fa640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85100a3350 0x7f85100a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.862+0000 7f8527fff640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85100a3350 0x7f85100a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.863+0000 7f85257fa640 1 -- 172.21.15.19:0/3568862232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f851c0cfea0 con 0x7f8528102790 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.863+0000 7f8527fff640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85100a3350 0x7f85100a5810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f851800bbe0 tx=0x7f851800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:06.866+0000 7f85257fa640 1 -- 172.21.15.19:0/3568862232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f851c014030 con 0x7f8528102790 2024-09-19T04:07:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.040+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8528062630 con 0x7f85100a3350 2024-09-19T04:07:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.049+0000 7f85257fa640 1 -- 172.21.15.19:0/3568862232 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340817 (secure 0 0 0) 0x7f8528062630 con 0x7f85100a3350 2024-09-19T04:07:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85100a3350 msgr2=0x7f85100a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85100a3350 0x7f85100a5810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f851800bbe0 tx=0x7f851800b390 comp rx=0 tx=0).stop 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528102790 msgr2=0x7f8528198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528102790 0x7f8528198880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c920 tx=0x7f851c00cde0 comp rx=0 tx=0).stop 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 shutdown_connections 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85100a3350 0x7f85100a5810 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8528106f10 0x7f852819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8528102790 0x7f8528198880 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 --2- 172.21.15.19:0/3568862232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8528101de0 0x7f8528198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 >> 172.21.15.19:0/3568862232 conn(0x7f85280fd8c0 msgr2=0x7f85280febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:07.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 shutdown_connections 2024-09-19T04:07:07.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:07.053+0000 7f852e7c3640 1 -- 172.21.15.19:0/3568862232 wait complete. 2024-09-19T04:07:07.717 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-19T04:07:06.445452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660499,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1182,"num_read_kb":714,"num_write":440,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1241696,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349436,"kb_used_data":12608,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592388,"statfs":{"total":1151940427776,"available":1151582605312,"internally_reserved":0,"allocated":12910592,"data_stored":8306249,"data_compressed":470438,"data_compressed_allocated":638976,"data_compressed_original":2809856,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":56,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002949"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:01.739932+0000","last_change":"2024-09-19T04:04:37.214769+0000","last_active":"2024-09-19T04:07:01.739932+0000","last_peered":"2024-09-19T04:07:01.739932+0000","last_clean":"2024-09-19T04:07:01.739932+0000","last_became_active":"2024-09-19T04:04:37.214518+0000","last_became_peered":"2024-09-19T04:04:37.214518+0000","last_unstale":"2024-09-19T04:07:01.739932+0000","last_undegraded":"2024-09-19T04:07:01.739932+0000","last_fullsized":"2024-09-19T04:07:01.739932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:50.704566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294660+0000","last_change":"2024-09-19T04:04:39.329733+0000","last_active":"2024-09-19T04:06:43.294660+0000","last_peered":"2024-09-19T04:06:43.294660+0000","last_clean":"2024-09-19T04:06:43.294660+0000","last_became_active":"2024-09-19T04:04:39.329031+0000","last_became_peered":"2024-09-19T04:04:39.329031+0000","last_unstale":"2024-09-19T04:06:43.294660+0000","last_undegraded":"2024-09-19T04:06:43.294660+0000","last_fullsized":"2024-09-19T04:06:43.294660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:57.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913948+0000","last_change":"2024-09-19T04:04:46.362679+0000","last_active":"2024-09-19T04:06:42.913948+0000","last_peered":"2024-09-19T04:06:42.913948+0000","last_clean":"2024-09-19T04:06:42.913948+0000","last_became_active":"2024-09-19T04:04:46.362257+0000","last_became_peered":"2024-09-19T04:04:46.362257+0000","last_unstale":"2024-09-19T04:06:42.913948+0000","last_undegraded":"2024-09-19T04:06:42.913948+0000","last_fullsized":"2024-09-19T04:06:42.913948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.389839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294777+0000","last_change":"2024-09-19T04:04:37.647295+0000","last_active":"2024-09-19T04:06:43.294777+0000","last_peered":"2024-09-19T04:06:43.294777+0000","last_clean":"2024-09-19T04:06:43.294777+0000","last_became_active":"2024-09-19T04:04:37.646964+0000","last_became_peered":"2024-09-19T04:04:37.646964+0000","last_unstale":"2024-09-19T04:06:43.294777+0000","last_undegraded":"2024-09-19T04:06:43.294777+0000","last_fullsized":"2024-09-19T04:06:43.294777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:49.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913163+0000","last_change":"2024-09-19T04:04:39.318270+0000","last_active":"2024-09-19T04:06:42.913163+0000","last_peered":"2024-09-19T04:06:42.913163+0000","last_clean":"2024-09-19T04:06:42.913163+0000","last_became_active":"2024-09-19T04:04:39.317972+0000","last_became_peered":"2024-09-19T04:04:39.317972+0000","last_unstale":"2024-09-19T04:06:42.913163+0000","last_undegraded":"2024-09-19T04:06:42.913163+0000","last_fullsized":"2024-09-19T04:06:42.913163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:25.994112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914875+0000","last_change":"2024-09-19T04:04:46.355111+0000","last_active":"2024-09-19T04:06:42.914875+0000","last_peered":"2024-09-19T04:06:42.914875+0000","last_clean":"2024-09-19T04:06:42.914875+0000","last_became_active":"2024-09-19T04:04:46.354684+0000","last_became_peered":"2024-09-19T04:04:46.354684+0000","last_unstale":"2024-09-19T04:06:42.914875+0000","last_undegraded":"2024-09-19T04:06:42.914875+0000","last_fullsized":"2024-09-19T04:06:42.914875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:33.141690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917054+0000","last_change":"2024-09-19T04:04:37.211772+0000","last_active":"2024-09-19T04:06:42.917054+0000","last_peered":"2024-09-19T04:06:42.917054+0000","last_clean":"2024-09-19T04:06:42.917054+0000","last_became_active":"2024-09-19T04:04:37.211452+0000","last_became_peered":"2024-09-19T04:04:37.211452+0000","last_unstale":"2024-09-19T04:06:42.917054+0000","last_undegraded":"2024-09-19T04:06:42.917054+0000","last_fullsized":"2024-09-19T04:06:42.917054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:57.781481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110814+0000","last_change":"2024-09-19T04:04:39.320791+0000","last_active":"2024-09-19T04:06:44.110814+0000","last_peered":"2024-09-19T04:06:44.110814+0000","last_clean":"2024-09-19T04:06:44.110814+0000","last_became_active":"2024-09-19T04:04:39.320206+0000","last_became_peered":"2024-09-19T04:04:39.320206+0000","last_unstale":"2024-09-19T04:06:44.110814+0000","last_undegraded":"2024-09-19T04:06:44.110814+0000","last_fullsized":"2024-09-19T04:06:44.110814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:17.385974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915320+0000","last_change":"2024-09-19T04:04:46.363375+0000","last_active":"2024-09-19T04:06:42.915320+0000","last_peered":"2024-09-19T04:06:42.915320+0000","last_clean":"2024-09-19T04:06:42.915320+0000","last_became_active":"2024-09-19T04:04:46.362800+0000","last_became_peered":"2024-09-19T04:04:46.362800+0000","last_unstale":"2024-09-19T04:06:42.915320+0000","last_undegraded":"2024-09-19T04:06:42.915320+0000","last_fullsized":"2024-09-19T04:06:42.915320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:55.651507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917026+0000","last_change":"2024-09-19T04:04:37.206218+0000","last_active":"2024-09-19T04:06:42.917026+0000","last_peered":"2024-09-19T04:06:42.917026+0000","last_clean":"2024-09-19T04:06:42.917026+0000","last_became_active":"2024-09-19T04:04:37.205675+0000","last_became_peered":"2024-09-19T04:04:37.205675+0000","last_unstale":"2024-09-19T04:06:42.917026+0000","last_undegraded":"2024-09-19T04:06:42.917026+0000","last_fullsized":"2024-09-19T04:06:42.917026+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:27.664960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912735+0000","last_change":"2024-09-19T04:04:39.329667+0000","last_active":"2024-09-19T04:06:42.912735+0000","last_peered":"2024-09-19T04:06:42.912735+0000","last_clean":"2024-09-19T04:06:42.912735+0000","last_became_active":"2024-09-19T04:04:39.328180+0000","last_became_peered":"2024-09-19T04:04:39.328180+0000","last_unstale":"2024-09-19T04:06:42.912735+0000","last_undegraded":"2024-09-19T04:06:42.912735+0000","last_fullsized":"2024-09-19T04:06:42.912735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:59.782472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914994+0000","last_change":"2024-09-19T04:04:46.359432+0000","last_active":"2024-09-19T04:06:42.914994+0000","last_peered":"2024-09-19T04:06:42.914994+0000","last_clean":"2024-09-19T04:06:42.914994+0000","last_became_active":"2024-09-19T04:04:46.358966+0000","last_became_peered":"2024-09-19T04:04:46.358966+0000","last_unstale":"2024-09-19T04:06:42.914994+0000","last_undegraded":"2024-09-19T04:06:42.914994+0000","last_fullsized":"2024-09-19T04:06:42.914994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:41.537926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418761+0000","last_change":"2024-09-19T04:04:39.325084+0000","last_active":"2024-09-19T04:06:43.418761+0000","last_peered":"2024-09-19T04:06:43.418761+0000","last_clean":"2024-09-19T04:06:43.418761+0000","last_became_active":"2024-09-19T04:04:39.324126+0000","last_became_peered":"2024-09-19T04:04:39.324126+0000","last_unstale":"2024-09-19T04:06:43.418761+0000","last_undegraded":"2024-09-19T04:06:43.418761+0000","last_fullsized":"2024-09-19T04:06:43.418761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:13.523889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110894+0000","last_change":"2024-09-19T04:05:42.359098+0000","last_active":"2024-09-19T04:06:44.110894+0000","last_peered":"2024-09-19T04:06:44.110894+0000","last_clean":"2024-09-19T04:06:44.110894+0000","last_became_active":"2024-09-19T04:05:42.341384+0000","last_became_peered":"2024-09-19T04:05:42.341384+0000","last_unstale":"2024-09-19T04:06:44.110894+0000","last_undegraded":"2024-09-19T04:06:44.110894+0000","last_fullsized":"2024-09-19T04:06:44.110894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:15:57.418355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915058+0000","last_change":"2024-09-19T04:04:46.374921+0000","last_active":"2024-09-19T04:06:42.915058+0000","last_peered":"2024-09-19T04:06:42.915058+0000","last_clean":"2024-09-19T04:06:42.915058+0000","last_became_active":"2024-09-19T04:04:46.374466+0000","last_became_peered":"2024-09-19T04:04:46.374466+0000","last_unstale":"2024-09-19T04:06:42.915058+0000","last_undegraded":"2024-09-19T04:06:42.915058+0000","last_fullsized":"2024-09-19T04:06:42.915058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:46.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294259+0000","last_change":"2024-09-19T04:04:39.319948+0000","last_active":"2024-09-19T04:06:43.294259+0000","last_peered":"2024-09-19T04:06:43.294259+0000","last_clean":"2024-09-19T04:06:43.294259+0000","last_became_active":"2024-09-19T04:04:39.319675+0000","last_became_peered":"2024-09-19T04:04:39.319675+0000","last_unstale":"2024-09-19T04:06:43.294259+0000","last_undegraded":"2024-09-19T04:06:43.294259+0000","last_fullsized":"2024-09-19T04:06:43.294259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:08.818133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914433+0000","last_change":"2024-09-19T04:04:37.207705+0000","last_active":"2024-09-19T04:06:42.914433+0000","last_peered":"2024-09-19T04:06:42.914433+0000","last_clean":"2024-09-19T04:06:42.914433+0000","last_became_active":"2024-09-19T04:04:37.207251+0000","last_became_peered":"2024-09-19T04:04:37.207251+0000","last_unstale":"2024-09-19T04:06:42.914433+0000","last_undegraded":"2024-09-19T04:06:42.914433+0000","last_fullsized":"2024-09-19T04:06:42.914433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:20.835004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914955+0000","last_change":"2024-09-19T04:04:46.354463+0000","last_active":"2024-09-19T04:06:42.914955+0000","last_peered":"2024-09-19T04:06:42.914955+0000","last_clean":"2024-09-19T04:06:42.914955+0000","last_became_active":"2024-09-19T04:04:46.353754+0000","last_became_peered":"2024-09-19T04:04:46.353754+0000","last_unstale":"2024-09-19T04:06:42.914955+0000","last_undegraded":"2024-09-19T04:06:42.914955+0000","last_fullsized":"2024-09-19T04:06:42.914955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:09.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110995+0000","last_change":"2024-09-19T04:04:39.322727+0000","last_active":"2024-09-19T04:06:44.110995+0000","last_peered":"2024-09-19T04:06:44.110995+0000","last_clean":"2024-09-19T04:06:44.110995+0000","last_became_active":"2024-09-19T04:04:39.320989+0000","last_became_peered":"2024-09-19T04:04:39.320989+0000","last_unstale":"2024-09-19T04:06:44.110995+0000","last_undegraded":"2024-09-19T04:06:44.110995+0000","last_fullsized":"2024-09-19T04:06:44.110995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:06.536540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294398+0000","last_change":"2024-09-19T04:04:37.213602+0000","last_active":"2024-09-19T04:06:43.294398+0000","last_peered":"2024-09-19T04:06:43.294398+0000","last_clean":"2024-09-19T04:06:43.294398+0000","last_became_active":"2024-09-19T04:04:37.213240+0000","last_became_peered":"2024-09-19T04:04:37.213240+0000","last_unstale":"2024-09-19T04:06:43.294398+0000","last_undegraded":"2024-09-19T04:06:43.294398+0000","last_fullsized":"2024-09-19T04:06:43.294398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:09.822393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913644+0000","last_change":"2024-09-19T04:04:46.363921+0000","last_active":"2024-09-19T04:06:42.913644+0000","last_peered":"2024-09-19T04:06:42.913644+0000","last_clean":"2024-09-19T04:06:42.913644+0000","last_became_active":"2024-09-19T04:04:46.363476+0000","last_became_peered":"2024-09-19T04:04:46.363476+0000","last_unstale":"2024-09-19T04:06:42.913644+0000","last_undegraded":"2024-09-19T04:06:42.913644+0000","last_fullsized":"2024-09-19T04:06:42.913644+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:32.015782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917044+0000","last_change":"2024-09-19T04:04:39.316814+0000","last_active":"2024-09-19T04:06:42.917044+0000","last_peered":"2024-09-19T04:06:42.917044+0000","last_clean":"2024-09-19T04:06:42.917044+0000","last_became_active":"2024-09-19T04:04:39.316504+0000","last_became_peered":"2024-09-19T04:04:39.316504+0000","last_unstale":"2024-09-19T04:06:42.917044+0000","last_undegraded":"2024-09-19T04:06:42.917044+0000","last_fullsized":"2024-09-19T04:06:42.917044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110786+0000","last_change":"2024-09-19T04:04:37.647260+0000","last_active":"2024-09-19T04:06:44.110786+0000","last_peered":"2024-09-19T04:06:44.110786+0000","last_clean":"2024-09-19T04:06:44.110786+0000","last_became_active":"2024-09-19T04:04:37.646971+0000","last_became_peered":"2024-09-19T04:04:37.646971+0000","last_unstale":"2024-09-19T04:06:44.110786+0000","last_undegraded":"2024-09-19T04:06:44.110786+0000","last_fullsized":"2024-09-19T04:06:44.110786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:07.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915112+0000","last_change":"2024-09-19T04:05:42.342340+0000","last_active":"2024-09-19T04:06:42.915112+0000","last_peered":"2024-09-19T04:06:42.915112+0000","last_clean":"2024-09-19T04:06:42.915112+0000","last_became_active":"2024-09-19T04:05:42.341861+0000","last_became_peered":"2024-09-19T04:05:42.341861+0000","last_unstale":"2024-09-19T04:06:42.915112+0000","last_undegraded":"2024-09-19T04:06:42.915112+0000","last_fullsized":"2024-09-19T04:06:42.915112+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:27.520514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916892+0000","last_change":"2024-09-19T04:06:42.916356+0000","last_active":"2024-09-19T04:06:42.916892+0000","last_peered":"2024-09-19T04:06:42.916892+0000","last_clean":"2024-09-19T04:06:42.916892+0000","last_became_active":"2024-09-19T04:06:42.915926+0000","last_became_peered":"2024-09-19T04:06:42.915926+0000","last_unstale":"2024-09-19T04:06:42.916892+0000","last_undegraded":"2024-09-19T04:06:42.916892+0000","last_fullsized":"2024-09-19T04:06:42.916892+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:46.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915360+0000","last_change":"2024-09-19T04:04:37.703740+0000","last_active":"2024-09-19T04:06:42.915360+0000","last_peered":"2024-09-19T04:06:42.915360+0000","last_clean":"2024-09-19T04:06:42.915360+0000","last_became_active":"2024-09-19T04:04:37.703414+0000","last_became_peered":"2024-09-19T04:04:37.703414+0000","last_unstale":"2024-09-19T04:06:42.915360+0000","last_undegraded":"2024-09-19T04:06:42.915360+0000","last_fullsized":"2024-09-19T04:06:42.915360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:19.653559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914409+0000","last_change":"2024-09-19T04:04:46.354964+0000","last_active":"2024-09-19T04:06:42.914409+0000","last_peered":"2024-09-19T04:06:42.914409+0000","last_clean":"2024-09-19T04:06:42.914409+0000","last_became_active":"2024-09-19T04:04:46.354494+0000","last_became_peered":"2024-09-19T04:04:46.354494+0000","last_unstale":"2024-09-19T04:06:42.914409+0000","last_undegraded":"2024-09-19T04:06:42.914409+0000","last_fullsized":"2024-09-19T04:06:42.914409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:11.246134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294858+0000","last_change":"2024-09-19T04:04:39.315811+0000","last_active":"2024-09-19T04:06:43.294858+0000","last_peered":"2024-09-19T04:06:43.294858+0000","last_clean":"2024-09-19T04:06:43.294858+0000","last_became_active":"2024-09-19T04:04:39.315466+0000","last_became_peered":"2024-09-19T04:04:39.315466+0000","last_unstale":"2024-09-19T04:06:43.294858+0000","last_undegraded":"2024-09-19T04:06:43.294858+0000","last_fullsized":"2024-09-19T04:06:43.294858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:17.414765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913659+0000","last_change":"2024-09-19T04:04:37.204206+0000","last_active":"2024-09-19T04:06:42.913659+0000","last_peered":"2024-09-19T04:06:42.913659+0000","last_clean":"2024-09-19T04:06:42.913659+0000","last_became_active":"2024-09-19T04:04:37.203845+0000","last_became_peered":"2024-09-19T04:04:37.203845+0000","last_unstale":"2024-09-19T04:06:42.913659+0000","last_undegraded":"2024-09-19T04:06:42.913659+0000","last_fullsized":"2024-09-19T04:06:42.913659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:52.868643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.293985+0000","last_change":"2024-09-19T04:04:46.375541+0000","last_active":"2024-09-19T04:06:43.293985+0000","last_peered":"2024-09-19T04:06:43.293985+0000","last_clean":"2024-09-19T04:06:43.293985+0000","last_became_active":"2024-09-19T04:04:46.375142+0000","last_became_peered":"2024-09-19T04:04:46.375142+0000","last_unstale":"2024-09-19T04:06:43.293985+0000","last_undegraded":"2024-09-19T04:06:43.293985+0000","last_fullsized":"2024-09-19T04:06:43.293985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.889459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909494+0000","last_change":"2024-09-19T04:04:39.324094+0000","last_active":"2024-09-19T04:06:42.909494+0000","last_peered":"2024-09-19T04:06:42.909494+0000","last_clean":"2024-09-19T04:06:42.909494+0000","last_became_active":"2024-09-19T04:04:39.322897+0000","last_became_peered":"2024-09-19T04:04:39.322897+0000","last_unstale":"2024-09-19T04:06:42.909494+0000","last_undegraded":"2024-09-19T04:06:42.909494+0000","last_fullsized":"2024-09-19T04:06:42.909494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:07.773110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917125+0000","last_change":"2024-09-19T04:04:37.647686+0000","last_active":"2024-09-19T04:06:42.917125+0000","last_peered":"2024-09-19T04:06:42.917125+0000","last_clean":"2024-09-19T04:06:42.917125+0000","last_became_active":"2024-09-19T04:04:37.646979+0000","last_became_peered":"2024-09-19T04:04:37.646979+0000","last_unstale":"2024-09-19T04:06:42.917125+0000","last_undegraded":"2024-09-19T04:06:42.917125+0000","last_fullsized":"2024-09-19T04:06:42.917125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:48.063075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111034+0000","last_change":"2024-09-19T04:04:46.356078+0000","last_active":"2024-09-19T04:06:44.111034+0000","last_peered":"2024-09-19T04:06:44.111034+0000","last_clean":"2024-09-19T04:06:44.111034+0000","last_became_active":"2024-09-19T04:04:46.355630+0000","last_became_peered":"2024-09-19T04:04:46.355630+0000","last_unstale":"2024-09-19T04:06:44.111034+0000","last_undegraded":"2024-09-19T04:06:44.111034+0000","last_fullsized":"2024-09-19T04:06:44.111034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:45.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914196+0000","last_change":"2024-09-19T04:04:39.311067+0000","last_active":"2024-09-19T04:06:42.914196+0000","last_peered":"2024-09-19T04:06:42.914196+0000","last_clean":"2024-09-19T04:06:42.914196+0000","last_became_active":"2024-09-19T04:04:39.310725+0000","last_became_peered":"2024-09-19T04:04:39.310725+0000","last_unstale":"2024-09-19T04:06:42.914196+0000","last_undegraded":"2024-09-19T04:06:42.914196+0000","last_fullsized":"2024-09-19T04:06:42.914196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:47:11.488121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913434+0000","last_change":"2024-09-19T04:04:37.648335+0000","last_active":"2024-09-19T04:06:42.913434+0000","last_peered":"2024-09-19T04:06:42.913434+0000","last_clean":"2024-09-19T04:06:42.913434+0000","last_became_active":"2024-09-19T04:04:37.648000+0000","last_became_peered":"2024-09-19T04:04:37.648000+0000","last_unstale":"2024-09-19T04:06:42.913434+0000","last_undegraded":"2024-09-19T04:06:42.913434+0000","last_fullsized":"2024-09-19T04:06:42.913434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:16.102728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916512+0000","last_change":"2024-09-19T04:04:46.375653+0000","last_active":"2024-09-19T04:06:42.916512+0000","last_peered":"2024-09-19T04:06:42.916512+0000","last_clean":"2024-09-19T04:06:42.916512+0000","last_became_active":"2024-09-19T04:04:46.375132+0000","last_became_peered":"2024-09-19T04:04:46.375132+0000","last_unstale":"2024-09-19T04:06:42.916512+0000","last_undegraded":"2024-09-19T04:06:42.916512+0000","last_fullsized":"2024-09-19T04:06:42.916512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:03.332697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914089+0000","last_change":"2024-09-19T04:04:39.318155+0000","last_active":"2024-09-19T04:06:42.914089+0000","last_peered":"2024-09-19T04:06:42.914089+0000","last_clean":"2024-09-19T04:06:42.914089+0000","last_became_active":"2024-09-19T04:04:39.316710+0000","last_became_peered":"2024-09-19T04:04:39.316710+0000","last_unstale":"2024-09-19T04:06:42.914089+0000","last_undegraded":"2024-09-19T04:06:42.914089+0000","last_fullsized":"2024-09-19T04:06:42.914089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:59.084523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:03.743187+0000","last_change":"2024-09-19T04:04:37.210695+0000","last_active":"2024-09-19T04:07:03.743187+0000","last_peered":"2024-09-19T04:07:03.743187+0000","last_clean":"2024-09-19T04:07:03.743187+0000","last_became_active":"2024-09-19T04:04:37.210334+0000","last_became_peered":"2024-09-19T04:04:37.210334+0000","last_unstale":"2024-09-19T04:07:03.743187+0000","last_undegraded":"2024-09-19T04:07:03.743187+0000","last_fullsized":"2024-09-19T04:07:03.743187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:12.545248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909313+0000","last_change":"2024-09-19T04:04:46.358130+0000","last_active":"2024-09-19T04:06:42.909313+0000","last_peered":"2024-09-19T04:06:42.909313+0000","last_clean":"2024-09-19T04:06:42.909313+0000","last_became_active":"2024-09-19T04:04:46.356810+0000","last_became_peered":"2024-09-19T04:04:46.356810+0000","last_unstale":"2024-09-19T04:06:42.909313+0000","last_undegraded":"2024-09-19T04:06:42.909313+0000","last_fullsized":"2024-09-19T04:06:42.909313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:49.884478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916817+0000","last_change":"2024-09-19T04:06:42.916354+0000","last_active":"2024-09-19T04:06:42.916817+0000","last_peered":"2024-09-19T04:06:42.916817+0000","last_clean":"2024-09-19T04:06:42.916817+0000","last_became_active":"2024-09-19T04:06:42.915943+0000","last_became_peered":"2024-09-19T04:06:42.915943+0000","last_unstale":"2024-09-19T04:06:42.916817+0000","last_undegraded":"2024-09-19T04:06:42.916817+0000","last_fullsized":"2024-09-19T04:06:42.916817+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:24.289049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:04.744026+0000","last_change":"2024-09-19T04:04:37.647310+0000","last_active":"2024-09-19T04:07:04.744026+0000","last_peered":"2024-09-19T04:07:04.744026+0000","last_clean":"2024-09-19T04:07:04.744026+0000","last_became_active":"2024-09-19T04:04:37.646903+0000","last_became_peered":"2024-09-19T04:04:37.646903+0000","last_unstale":"2024-09-19T04:07:04.744026+0000","last_undegraded":"2024-09-19T04:07:04.744026+0000","last_fullsized":"2024-09-19T04:07:04.744026+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:20.097472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909319+0000","last_change":"2024-09-19T04:04:46.375261+0000","last_active":"2024-09-19T04:06:42.909319+0000","last_peered":"2024-09-19T04:06:42.909319+0000","last_clean":"2024-09-19T04:06:42.909319+0000","last_became_active":"2024-09-19T04:04:46.374489+0000","last_became_peered":"2024-09-19T04:04:46.374489+0000","last_unstale":"2024-09-19T04:06:42.909319+0000","last_undegraded":"2024-09-19T04:06:42.909319+0000","last_fullsized":"2024-09-19T04:06:42.909319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:05.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.908952+0000","last_change":"2024-09-19T04:04:39.322265+0000","last_active":"2024-09-19T04:06:42.908952+0000","last_peered":"2024-09-19T04:06:42.908952+0000","last_clean":"2024-09-19T04:06:42.908952+0000","last_became_active":"2024-09-19T04:04:39.321938+0000","last_became_peered":"2024-09-19T04:04:39.321938+0000","last_unstale":"2024-09-19T04:06:42.908952+0000","last_undegraded":"2024-09-19T04:06:42.908952+0000","last_fullsized":"2024-09-19T04:06:42.908952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:19.220661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418797+0000","last_change":"2024-09-19T04:04:37.211330+0000","last_active":"2024-09-19T04:06:43.418797+0000","last_peered":"2024-09-19T04:06:43.418797+0000","last_clean":"2024-09-19T04:06:43.418797+0000","last_became_active":"2024-09-19T04:04:37.210788+0000","last_became_peered":"2024-09-19T04:04:37.210788+0000","last_unstale":"2024-09-19T04:06:43.418797+0000","last_undegraded":"2024-09-19T04:06:43.418797+0000","last_fullsized":"2024-09-19T04:06:43.418797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:43.169897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913106+0000","last_change":"2024-09-19T04:05:42.465969+0000","last_active":"2024-09-19T04:06:42.913106+0000","last_peered":"2024-09-19T04:06:42.913106+0000","last_clean":"2024-09-19T04:06:42.913106+0000","last_became_active":"2024-09-19T04:05:42.465621+0000","last_became_peered":"2024-09-19T04:05:42.465621+0000","last_unstale":"2024-09-19T04:06:42.913106+0000","last_undegraded":"2024-09-19T04:06:42.913106+0000","last_fullsized":"2024-09-19T04:06:42.913106+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:29.802141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,10],"acting":[5,1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294450+0000","last_change":"2024-09-19T04:04:39.329480+0000","last_active":"2024-09-19T04:06:43.294450+0000","last_peered":"2024-09-19T04:06:43.294450+0000","last_clean":"2024-09-19T04:06:43.294450+0000","last_became_active":"2024-09-19T04:04:39.329008+0000","last_became_peered":"2024-09-19T04:04:39.329008+0000","last_unstale":"2024-09-19T04:06:43.294450+0000","last_undegraded":"2024-09-19T04:06:43.294450+0000","last_fullsized":"2024-09-19T04:06:43.294450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:06.782579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915484+0000","last_change":"2024-09-19T04:04:37.216224+0000","last_active":"2024-09-19T04:06:42.915484+0000","last_peered":"2024-09-19T04:06:42.915484+0000","last_clean":"2024-09-19T04:06:42.915484+0000","last_became_active":"2024-09-19T04:04:37.215846+0000","last_became_peered":"2024-09-19T04:04:37.215846+0000","last_unstale":"2024-09-19T04:06:42.915484+0000","last_undegraded":"2024-09-19T04:06:42.915484+0000","last_fullsized":"2024-09-19T04:06:42.915484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:57.829694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418966+0000","last_change":"2024-09-19T04:03:25.869634+0000","last_active":"2024-09-19T04:06:43.418966+0000","last_peered":"2024-09-19T04:06:43.418966+0000","last_clean":"2024-09-19T04:06:43.418966+0000","last_became_active":"2024-09-19T04:03:25.869250+0000","last_became_peered":"2024-09-19T04:03:25.869250+0000","last_unstale":"2024-09-19T04:06:43.418966+0000","last_undegraded":"2024-09-19T04:06:43.418966+0000","last_fullsized":"2024-09-19T04:06:43.418966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_clean_scrub_stamp":"2024-09-19T03:59:59.253417+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:51.956932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419061+0000","last_change":"2024-09-19T04:04:46.357123+0000","last_active":"2024-09-19T04:06:43.419061+0000","last_peered":"2024-09-19T04:06:43.419061+0000","last_clean":"2024-09-19T04:06:43.419061+0000","last_became_active":"2024-09-19T04:04:46.356671+0000","last_became_peered":"2024-09-19T04:04:46.356671+0000","last_unstale":"2024-09-19T04:06:43.419061+0000","last_undegraded":"2024-09-19T04:06:43.419061+0000","last_fullsized":"2024-09-19T04:06:43.419061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:09.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419189+0000","last_change":"2024-09-19T04:04:39.322487+0000","last_active":"2024-09-19T04:06:43.419189+0000","last_peered":"2024-09-19T04:06:43.419189+0000","last_clean":"2024-09-19T04:06:43.419189+0000","last_became_active":"2024-09-19T04:04:39.321994+0000","last_became_peered":"2024-09-19T04:04:39.321994+0000","last_unstale":"2024-09-19T04:06:43.419189+0000","last_undegraded":"2024-09-19T04:06:43.419189+0000","last_fullsized":"2024-09-19T04:06:43.419189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:56.977405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":147,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:01.750953+0000","last_change":"2024-09-19T04:04:37.703452+0000","last_active":"2024-09-19T04:07:01.750953+0000","last_peered":"2024-09-19T04:07:01.750953+0000","last_clean":"2024-09-19T04:07:01.750953+0000","last_became_active":"2024-09-19T04:04:37.703048+0000","last_became_peered":"2024-09-19T04:04:37.703048+0000","last_unstale":"2024-09-19T04:07:01.750953+0000","last_undegraded":"2024-09-19T04:07:01.750953+0000","last_fullsized":"2024-09-19T04:07:01.750953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:16:29.051617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110726+0000","last_change":"2024-09-19T04:04:46.361229+0000","last_active":"2024-09-19T04:06:44.110726+0000","last_peered":"2024-09-19T04:06:44.110726+0000","last_clean":"2024-09-19T04:06:44.110726+0000","last_became_active":"2024-09-19T04:04:46.360729+0000","last_became_peered":"2024-09-19T04:04:46.360729+0000","last_unstale":"2024-09-19T04:06:44.110726+0000","last_undegraded":"2024-09-19T04:06:44.110726+0000","last_fullsized":"2024-09-19T04:06:44.110726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:48.022778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418940+0000","last_change":"2024-09-19T04:04:39.328598+0000","last_active":"2024-09-19T04:06:43.418940+0000","last_peered":"2024-09-19T04:06:43.418940+0000","last_clean":"2024-09-19T04:06:43.418940+0000","last_became_active":"2024-09-19T04:04:39.328247+0000","last_became_peered":"2024-09-19T04:04:39.328247+0000","last_unstale":"2024-09-19T04:06:43.418940+0000","last_undegraded":"2024-09-19T04:06:43.418940+0000","last_fullsized":"2024-09-19T04:06:43.418940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:34.916487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913045+0000","last_change":"2024-09-19T04:04:37.212065+0000","last_active":"2024-09-19T04:06:42.913045+0000","last_peered":"2024-09-19T04:06:42.913045+0000","last_clean":"2024-09-19T04:06:42.913045+0000","last_became_active":"2024-09-19T04:04:37.211520+0000","last_became_peered":"2024-09-19T04:04:37.211520+0000","last_unstale":"2024-09-19T04:06:42.913045+0000","last_undegraded":"2024-09-19T04:06:42.913045+0000","last_fullsized":"2024-09-19T04:06:42.913045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:53.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912491+0000","last_change":"2024-09-19T04:04:46.358558+0000","last_active":"2024-09-19T04:06:42.912491+0000","last_peered":"2024-09-19T04:06:42.912491+0000","last_clean":"2024-09-19T04:06:42.912491+0000","last_became_active":"2024-09-19T04:04:46.358158+0000","last_became_peered":"2024-09-19T04:04:46.358158+0000","last_unstale":"2024-09-19T04:06:42.912491+0000","last_undegraded":"2024-09-19T04:06:42.912491+0000","last_fullsized":"2024-09-19T04:06:42.912491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:16.627051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419310+0000","last_change":"2024-09-19T04:04:39.325187+0000","last_active":"2024-09-19T04:06:43.419310+0000","last_peered":"2024-09-19T04:06:43.419310+0000","last_clean":"2024-09-19T04:06:43.419310+0000","last_became_active":"2024-09-19T04:04:39.324644+0000","last_became_peered":"2024-09-19T04:04:39.324644+0000","last_unstale":"2024-09-19T04:06:43.419310+0000","last_undegraded":"2024-09-19T04:06:43.419310+0000","last_fullsized":"2024-09-19T04:06:43.419310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:11.879273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909362+0000","last_change":"2024-09-19T04:04:37.205389+0000","last_active":"2024-09-19T04:06:42.909362+0000","last_peered":"2024-09-19T04:06:42.909362+0000","last_clean":"2024-09-19T04:06:42.909362+0000","last_became_active":"2024-09-19T04:04:37.205117+0000","last_became_peered":"2024-09-19T04:04:37.205117+0000","last_unstale":"2024-09-19T04:06:42.909362+0000","last_undegraded":"2024-09-19T04:06:42.909362+0000","last_fullsized":"2024-09-19T04:06:42.909362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:49.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916378+0000","last_change":"2024-09-19T04:05:42.342246+0000","last_active":"2024-09-19T04:06:42.916378+0000","last_peered":"2024-09-19T04:06:42.916378+0000","last_clean":"2024-09-19T04:06:42.916378+0000","last_became_active":"2024-09-19T04:05:42.341628+0000","last_became_peered":"2024-09-19T04:05:42.341628+0000","last_unstale":"2024-09-19T04:06:42.916378+0000","last_undegraded":"2024-09-19T04:06:42.916378+0000","last_fullsized":"2024-09-19T04:06:42.916378+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:16.637647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111363+0000","last_change":"2024-09-19T04:04:39.328841+0000","last_active":"2024-09-19T04:06:44.111363+0000","last_peered":"2024-09-19T04:06:44.111363+0000","last_clean":"2024-09-19T04:06:44.111363+0000","last_became_active":"2024-09-19T04:04:39.328470+0000","last_became_peered":"2024-09-19T04:04:39.328470+0000","last_unstale":"2024-09-19T04:06:44.111363+0000","last_undegraded":"2024-09-19T04:06:44.111363+0000","last_fullsized":"2024-09-19T04:06:44.111363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:17.701469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111344+0000","last_change":"2024-09-19T04:05:42.353456+0000","last_active":"2024-09-19T04:06:44.111344+0000","last_peered":"2024-09-19T04:06:44.111344+0000","last_clean":"2024-09-19T04:06:44.111344+0000","last_became_active":"2024-09-19T04:05:42.340710+0000","last_became_peered":"2024-09-19T04:05:42.340710+0000","last_unstale":"2024-09-19T04:06:44.111344+0000","last_undegraded":"2024-09-19T04:06:44.111344+0000","last_fullsized":"2024-09-19T04:06:44.111344+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:45.262693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912021+0000","last_change":"2024-09-19T04:04:46.361199+0000","last_active":"2024-09-19T04:06:42.912021+0000","last_peered":"2024-09-19T04:06:42.912021+0000","last_clean":"2024-09-19T04:06:42.912021+0000","last_became_active":"2024-09-19T04:04:46.360639+0000","last_became_peered":"2024-09-19T04:04:46.360639+0000","last_unstale":"2024-09-19T04:06:42.912021+0000","last_undegraded":"2024-09-19T04:06:42.912021+0000","last_fullsized":"2024-09-19T04:06:42.912021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:50.185760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912115+0000","last_change":"2024-09-19T04:04:39.328529+0000","last_active":"2024-09-19T04:06:42.912115+0000","last_peered":"2024-09-19T04:06:42.912115+0000","last_clean":"2024-09-19T04:06:42.912115+0000","last_became_active":"2024-09-19T04:04:39.328174+0000","last_became_peered":"2024-09-19T04:04:39.328174+0000","last_unstale":"2024-09-19T04:06:42.912115+0000","last_undegraded":"2024-09-19T04:06:42.912115+0000","last_fullsized":"2024-09-19T04:06:42.912115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:02.503672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916702+0000","last_change":"2024-09-19T04:04:37.647919+0000","last_active":"2024-09-19T04:06:42.916702+0000","last_peered":"2024-09-19T04:06:42.916702+0000","last_clean":"2024-09-19T04:06:42.916702+0000","last_became_active":"2024-09-19T04:04:37.647305+0000","last_became_peered":"2024-09-19T04:04:37.647305+0000","last_unstale":"2024-09-19T04:06:42.916702+0000","last_undegraded":"2024-09-19T04:06:42.916702+0000","last_fullsized":"2024-09-19T04:06:42.916702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:07.035465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909285+0000","last_change":"2024-09-19T04:04:46.360362+0000","last_active":"2024-09-19T04:06:42.909285+0000","last_peered":"2024-09-19T04:06:42.909285+0000","last_clean":"2024-09-19T04:06:42.909285+0000","last_became_active":"2024-09-19T04:04:46.359470+0000","last_became_peered":"2024-09-19T04:04:46.359470+0000","last_unstale":"2024-09-19T04:06:42.909285+0000","last_undegraded":"2024-09-19T04:06:42.909285+0000","last_fullsized":"2024-09-19T04:06:42.909285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:43.804073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418669+0000","last_change":"2024-09-19T04:04:39.326090+0000","last_active":"2024-09-19T04:06:43.418669+0000","last_peered":"2024-09-19T04:06:43.418669+0000","last_clean":"2024-09-19T04:06:43.418669+0000","last_became_active":"2024-09-19T04:04:39.325751+0000","last_became_peered":"2024-09-19T04:04:39.325751+0000","last_unstale":"2024-09-19T04:06:43.418669+0000","last_undegraded":"2024-09-19T04:06:43.418669+0000","last_fullsized":"2024-09-19T04:06:43.418669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:25.357467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:03.743096+0000","last_change":"2024-09-19T04:04:37.215107+0000","last_active":"2024-09-19T04:07:03.743096+0000","last_peered":"2024-09-19T04:07:03.743096+0000","last_clean":"2024-09-19T04:07:03.743096+0000","last_became_active":"2024-09-19T04:04:37.214693+0000","last_became_peered":"2024-09-19T04:04:37.214693+0000","last_unstale":"2024-09-19T04:07:03.743096+0000","last_undegraded":"2024-09-19T04:07:03.743096+0000","last_fullsized":"2024-09-19T04:07:03.743096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:56.535316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913700+0000","last_change":"2024-09-19T04:04:46.363625+0000","last_active":"2024-09-19T04:06:42.913700+0000","last_peered":"2024-09-19T04:06:42.913700+0000","last_clean":"2024-09-19T04:06:42.913700+0000","last_became_active":"2024-09-19T04:04:46.363207+0000","last_became_peered":"2024-09-19T04:04:46.363207+0000","last_unstale":"2024-09-19T04:06:42.913700+0000","last_undegraded":"2024-09-19T04:06:42.913700+0000","last_fullsized":"2024-09-19T04:06:42.913700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:30.251168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418748+0000","last_change":"2024-09-19T04:05:42.466411+0000","last_active":"2024-09-19T04:06:43.418748+0000","last_peered":"2024-09-19T04:06:43.418748+0000","last_clean":"2024-09-19T04:06:43.418748+0000","last_became_active":"2024-09-19T04:05:42.465999+0000","last_became_peered":"2024-09-19T04:05:42.465999+0000","last_unstale":"2024-09-19T04:06:43.418748+0000","last_undegraded":"2024-09-19T04:06:43.418748+0000","last_fullsized":"2024-09-19T04:06:43.418748+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:51.289726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912476+0000","last_change":"2024-09-19T04:04:37.212103+0000","last_active":"2024-09-19T04:06:42.912476+0000","last_peered":"2024-09-19T04:06:42.912476+0000","last_clean":"2024-09-19T04:06:42.912476+0000","last_became_active":"2024-09-19T04:04:37.211739+0000","last_became_peered":"2024-09-19T04:04:37.211739+0000","last_unstale":"2024-09-19T04:06:42.912476+0000","last_undegraded":"2024-09-19T04:06:42.912476+0000","last_fullsized":"2024-09-19T04:06:42.912476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:06.588194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110915+0000","last_change":"2024-09-19T04:04:46.361975+0000","last_active":"2024-09-19T04:06:44.110915+0000","last_peered":"2024-09-19T04:06:44.110915+0000","last_clean":"2024-09-19T04:06:44.110915+0000","last_became_active":"2024-09-19T04:04:46.361492+0000","last_became_peered":"2024-09-19T04:04:46.361492+0000","last_unstale":"2024-09-19T04:06:44.110915+0000","last_undegraded":"2024-09-19T04:06:44.110915+0000","last_fullsized":"2024-09-19T04:06:44.110915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:52.723836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909144+0000","last_change":"2024-09-19T04:04:39.317756+0000","last_active":"2024-09-19T04:06:42.909144+0000","last_peered":"2024-09-19T04:06:42.909144+0000","last_clean":"2024-09-19T04:06:42.909144+0000","last_became_active":"2024-09-19T04:04:39.317322+0000","last_became_peered":"2024-09-19T04:04:39.317322+0000","last_unstale":"2024-09-19T04:06:42.909144+0000","last_undegraded":"2024-09-19T04:06:42.909144+0000","last_fullsized":"2024-09-19T04:06:42.909144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:26.291574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914459+0000","last_change":"2024-09-19T04:04:37.213892+0000","last_active":"2024-09-19T04:06:42.914459+0000","last_peered":"2024-09-19T04:06:42.914459+0000","last_clean":"2024-09-19T04:06:42.914459+0000","last_became_active":"2024-09-19T04:04:37.213350+0000","last_became_peered":"2024-09-19T04:04:37.213350+0000","last_unstale":"2024-09-19T04:06:42.914459+0000","last_undegraded":"2024-09-19T04:06:42.914459+0000","last_fullsized":"2024-09-19T04:06:42.914459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:40.065717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912727+0000","last_change":"2024-09-19T04:04:46.360566+0000","last_active":"2024-09-19T04:06:42.912727+0000","last_peered":"2024-09-19T04:06:42.912727+0000","last_clean":"2024-09-19T04:06:42.912727+0000","last_became_active":"2024-09-19T04:04:46.360143+0000","last_became_peered":"2024-09-19T04:04:46.360143+0000","last_unstale":"2024-09-19T04:06:42.912727+0000","last_undegraded":"2024-09-19T04:06:42.912727+0000","last_fullsized":"2024-09-19T04:06:42.912727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:59.997088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916894+0000","last_change":"2024-09-19T04:04:39.318363+0000","last_active":"2024-09-19T04:06:42.916894+0000","last_peered":"2024-09-19T04:06:42.916894+0000","last_clean":"2024-09-19T04:06:42.916894+0000","last_became_active":"2024-09-19T04:04:39.318003+0000","last_became_peered":"2024-09-19T04:04:39.318003+0000","last_unstale":"2024-09-19T04:06:42.916894+0000","last_undegraded":"2024-09-19T04:06:42.916894+0000","last_fullsized":"2024-09-19T04:06:42.916894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:54.128795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:02.773578+0000","last_change":"2024-09-19T04:05:42.349944+0000","last_active":"2024-09-19T04:07:02.773578+0000","last_peered":"2024-09-19T04:07:02.773578+0000","last_clean":"2024-09-19T04:07:02.773578+0000","last_became_active":"2024-09-19T04:05:42.339605+0000","last_became_peered":"2024-09-19T04:05:42.339605+0000","last_unstale":"2024-09-19T04:07:02.773578+0000","last_undegraded":"2024-09-19T04:07:02.773578+0000","last_fullsized":"2024-09-19T04:07:02.773578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:17.508316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62633,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":168,"num_write":41,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":60938,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912879+0000","last_change":"2024-09-19T04:04:46.362967+0000","last_active":"2024-09-19T04:06:42.912879+0000","last_peered":"2024-09-19T04:06:42.912879+0000","last_clean":"2024-09-19T04:06:42.912879+0000","last_became_active":"2024-09-19T04:04:46.362529+0000","last_became_peered":"2024-09-19T04:04:46.362529+0000","last_unstale":"2024-09-19T04:06:42.912879+0000","last_undegraded":"2024-09-19T04:06:42.912879+0000","last_fullsized":"2024-09-19T04:06:42.912879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:01.821697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110944+0000","last_change":"2024-09-19T04:04:39.323499+0000","last_active":"2024-09-19T04:06:44.110944+0000","last_peered":"2024-09-19T04:06:44.110944+0000","last_clean":"2024-09-19T04:06:44.110944+0000","last_became_active":"2024-09-19T04:04:39.323076+0000","last_became_peered":"2024-09-19T04:04:39.323076+0000","last_unstale":"2024-09-19T04:06:44.110944+0000","last_undegraded":"2024-09-19T04:06:44.110944+0000","last_fullsized":"2024-09-19T04:06:44.110944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:10.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913160+0000","last_change":"2024-09-19T04:04:37.647927+0000","last_active":"2024-09-19T04:06:42.913160+0000","last_peered":"2024-09-19T04:06:42.913160+0000","last_clean":"2024-09-19T04:06:42.913160+0000","last_became_active":"2024-09-19T04:04:37.647279+0000","last_became_peered":"2024-09-19T04:04:37.647279+0000","last_unstale":"2024-09-19T04:06:42.913160+0000","last_undegraded":"2024-09-19T04:06:42.913160+0000","last_fullsized":"2024-09-19T04:06:42.913160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:05:50.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914938+0000","last_change":"2024-09-19T04:04:46.363562+0000","last_active":"2024-09-19T04:06:42.914938+0000","last_peered":"2024-09-19T04:06:42.914938+0000","last_clean":"2024-09-19T04:06:42.914938+0000","last_became_active":"2024-09-19T04:04:46.362915+0000","last_became_peered":"2024-09-19T04:04:46.362915+0000","last_unstale":"2024-09-19T04:06:42.914938+0000","last_undegraded":"2024-09-19T04:06:42.914938+0000","last_fullsized":"2024-09-19T04:06:42.914938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:08.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.420917+0000","last_change":"2024-09-19T04:06:43.420595+0000","last_active":"2024-09-19T04:06:43.420917+0000","last_peered":"2024-09-19T04:06:43.420917+0000","last_clean":"2024-09-19T04:06:43.420917+0000","last_became_active":"2024-09-19T04:06:43.420316+0000","last_became_peered":"2024-09-19T04:06:43.420316+0000","last_unstale":"2024-09-19T04:06:43.420917+0000","last_undegraded":"2024-09-19T04:06:43.420917+0000","last_fullsized":"2024-09-19T04:06:43.420917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:12:54.798028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419119+0000","last_change":"2024-09-19T04:04:37.212393+0000","last_active":"2024-09-19T04:06:43.419119+0000","last_peered":"2024-09-19T04:06:43.419119+0000","last_clean":"2024-09-19T04:06:43.419119+0000","last_became_active":"2024-09-19T04:04:37.212013+0000","last_became_peered":"2024-09-19T04:04:37.212013+0000","last_unstale":"2024-09-19T04:06:43.419119+0000","last_undegraded":"2024-09-19T04:06:43.419119+0000","last_fullsized":"2024-09-19T04:06:43.419119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:37.981075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916368+0000","last_change":"2024-09-19T04:05:43.151378+0000","last_active":"2024-09-19T04:06:42.916368+0000","last_peered":"2024-09-19T04:06:42.916368+0000","last_clean":"2024-09-19T04:06:42.916368+0000","last_became_active":"2024-09-19T04:05:43.150907+0000","last_became_peered":"2024-09-19T04:05:43.150907+0000","last_unstale":"2024-09-19T04:06:42.916368+0000","last_undegraded":"2024-09-19T04:06:42.916368+0000","last_fullsized":"2024-09-19T04:06:42.916368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:09.542371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111185+0000","last_change":"2024-09-19T04:04:39.319705+0000","last_active":"2024-09-19T04:06:44.111185+0000","last_peered":"2024-09-19T04:06:44.111185+0000","last_clean":"2024-09-19T04:06:44.111185+0000","last_became_active":"2024-09-19T04:04:39.318065+0000","last_became_peered":"2024-09-19T04:04:39.318065+0000","last_unstale":"2024-09-19T04:06:44.111185+0000","last_undegraded":"2024-09-19T04:06:44.111185+0000","last_fullsized":"2024-09-19T04:06:44.111185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:15.630927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":246,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:02.770896+0000","last_change":"2024-09-19T04:04:37.215192+0000","last_active":"2024-09-19T04:07:02.770896+0000","last_peered":"2024-09-19T04:07:02.770896+0000","last_clean":"2024-09-19T04:07:02.770896+0000","last_became_active":"2024-09-19T04:04:37.214872+0000","last_became_peered":"2024-09-19T04:04:37.214872+0000","last_unstale":"2024-09-19T04:07:02.770896+0000","last_undegraded":"2024-09-19T04:07:02.770896+0000","last_fullsized":"2024-09-19T04:07:02.770896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:18.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294101+0000","last_change":"2024-09-19T04:04:46.358113+0000","last_active":"2024-09-19T04:06:43.294101+0000","last_peered":"2024-09-19T04:06:43.294101+0000","last_clean":"2024-09-19T04:06:43.294101+0000","last_became_active":"2024-09-19T04:04:46.356604+0000","last_became_peered":"2024-09-19T04:04:46.356604+0000","last_unstale":"2024-09-19T04:06:43.294101+0000","last_undegraded":"2024-09-19T04:06:43.294101+0000","last_fullsized":"2024-09-19T04:06:43.294101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:38.147738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912583+0000","last_change":"2024-09-19T04:04:46.354516+0000","last_active":"2024-09-19T04:06:42.912583+0000","last_peered":"2024-09-19T04:06:42.912583+0000","last_clean":"2024-09-19T04:06:42.912583+0000","last_became_active":"2024-09-19T04:04:46.354034+0000","last_became_peered":"2024-09-19T04:04:46.354034+0000","last_unstale":"2024-09-19T04:06:42.912583+0000","last_undegraded":"2024-09-19T04:06:42.912583+0000","last_fullsized":"2024-09-19T04:06:42.912583+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:21.946746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912869+0000","last_change":"2024-09-19T04:04:39.311324+0000","last_active":"2024-09-19T04:06:42.912869+0000","last_peered":"2024-09-19T04:06:42.912869+0000","last_clean":"2024-09-19T04:06:42.912869+0000","last_became_active":"2024-09-19T04:04:39.310977+0000","last_became_peered":"2024-09-19T04:04:39.310977+0000","last_unstale":"2024-09-19T04:06:42.912869+0000","last_undegraded":"2024-09-19T04:06:42.912869+0000","last_fullsized":"2024-09-19T04:06:42.912869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:43.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912352+0000","last_change":"2024-09-19T04:04:37.212773+0000","last_active":"2024-09-19T04:06:42.912352+0000","last_peered":"2024-09-19T04:06:42.912352+0000","last_clean":"2024-09-19T04:06:42.912352+0000","last_became_active":"2024-09-19T04:04:37.212405+0000","last_became_peered":"2024-09-19T04:04:37.212405+0000","last_unstale":"2024-09-19T04:06:42.912352+0000","last_undegraded":"2024-09-19T04:06:42.912352+0000","last_fullsized":"2024-09-19T04:06:42.912352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:29.020122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111297+0000","last_change":"2024-09-19T04:04:46.363052+0000","last_active":"2024-09-19T04:06:44.111297+0000","last_peered":"2024-09-19T04:06:44.111297+0000","last_clean":"2024-09-19T04:06:44.111297+0000","last_became_active":"2024-09-19T04:04:46.362588+0000","last_became_peered":"2024-09-19T04:04:46.362588+0000","last_unstale":"2024-09-19T04:06:44.111297+0000","last_undegraded":"2024-09-19T04:06:44.111297+0000","last_fullsized":"2024-09-19T04:06:44.111297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:52.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294114+0000","last_change":"2024-09-19T04:04:39.313686+0000","last_active":"2024-09-19T04:06:43.294114+0000","last_peered":"2024-09-19T04:06:43.294114+0000","last_clean":"2024-09-19T04:06:43.294114+0000","last_became_active":"2024-09-19T04:04:39.313386+0000","last_became_peered":"2024-09-19T04:04:39.313386+0000","last_unstale":"2024-09-19T04:06:43.294114+0000","last_undegraded":"2024-09-19T04:06:43.294114+0000","last_fullsized":"2024-09-19T04:06:43.294114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:08.274930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.908950+0000","last_change":"2024-09-19T04:04:37.207926+0000","last_active":"2024-09-19T04:06:42.908950+0000","last_peered":"2024-09-19T04:06:42.908950+0000","last_clean":"2024-09-19T04:06:42.908950+0000","last_became_active":"2024-09-19T04:04:37.207538+0000","last_became_peered":"2024-09-19T04:04:37.207538+0000","last_unstale":"2024-09-19T04:06:42.908950+0000","last_undegraded":"2024-09-19T04:06:42.908950+0000","last_fullsized":"2024-09-19T04:06:42.908950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:29.621638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913935+0000","last_change":"2024-09-19T04:04:37.212829+0000","last_active":"2024-09-19T04:06:42.913935+0000","last_peered":"2024-09-19T04:06:42.913935+0000","last_clean":"2024-09-19T04:06:42.913935+0000","last_became_active":"2024-09-19T04:04:37.212294+0000","last_became_peered":"2024-09-19T04:04:37.212294+0000","last_unstale":"2024-09-19T04:06:42.913935+0000","last_undegraded":"2024-09-19T04:06:42.913935+0000","last_fullsized":"2024-09-19T04:06:42.913935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:34.479708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912531+0000","last_change":"2024-09-19T04:04:39.328622+0000","last_active":"2024-09-19T04:06:42.912531+0000","last_peered":"2024-09-19T04:06:42.912531+0000","last_clean":"2024-09-19T04:06:42.912531+0000","last_became_active":"2024-09-19T04:04:39.328290+0000","last_became_peered":"2024-09-19T04:04:39.328290+0000","last_unstale":"2024-09-19T04:06:42.912531+0000","last_undegraded":"2024-09-19T04:06:42.912531+0000","last_fullsized":"2024-09-19T04:06:42.912531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:04.290767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914155+0000","last_change":"2024-09-19T04:04:46.362853+0000","last_active":"2024-09-19T04:06:42.914155+0000","last_peered":"2024-09-19T04:06:42.914155+0000","last_clean":"2024-09-19T04:06:42.914155+0000","last_became_active":"2024-09-19T04:04:46.362392+0000","last_became_peered":"2024-09-19T04:04:46.362392+0000","last_unstale":"2024-09-19T04:06:42.914155+0000","last_undegraded":"2024-09-19T04:06:42.914155+0000","last_fullsized":"2024-09-19T04:06:42.914155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:40.016502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:03.753495+0000","last_change":"2024-09-19T04:04:37.213248+0000","last_active":"2024-09-19T04:07:03.753495+0000","last_peered":"2024-09-19T04:07:03.753495+0000","last_clean":"2024-09-19T04:07:03.753495+0000","last_became_active":"2024-09-19T04:04:37.212639+0000","last_became_peered":"2024-09-19T04:04:37.212639+0000","last_unstale":"2024-09-19T04:07:03.753495+0000","last_undegraded":"2024-09-19T04:07:03.753495+0000","last_fullsized":"2024-09-19T04:07:03.753495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:45:05.917792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916668+0000","last_change":"2024-09-19T04:04:39.330160+0000","last_active":"2024-09-19T04:06:42.916668+0000","last_peered":"2024-09-19T04:06:42.916668+0000","last_clean":"2024-09-19T04:06:42.916668+0000","last_became_active":"2024-09-19T04:04:39.329828+0000","last_became_peered":"2024-09-19T04:04:39.329828+0000","last_unstale":"2024-09-19T04:06:42.916668+0000","last_undegraded":"2024-09-19T04:06:42.916668+0000","last_fullsized":"2024-09-19T04:06:42.916668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:57.712550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914137+0000","last_change":"2024-09-19T04:04:46.360459+0000","last_active":"2024-09-19T04:06:42.914137+0000","last_peered":"2024-09-19T04:06:42.914137+0000","last_clean":"2024-09-19T04:06:42.914137+0000","last_became_active":"2024-09-19T04:04:46.359914+0000","last_became_peered":"2024-09-19T04:04:46.359914+0000","last_unstale":"2024-09-19T04:06:42.914137+0000","last_undegraded":"2024-09-19T04:06:42.914137+0000","last_fullsized":"2024-09-19T04:06:42.914137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:30.380358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110927+0000","last_change":"2024-09-19T04:04:39.324847+0000","last_active":"2024-09-19T04:06:44.110927+0000","last_peered":"2024-09-19T04:06:44.110927+0000","last_clean":"2024-09-19T04:06:44.110927+0000","last_became_active":"2024-09-19T04:04:39.324402+0000","last_became_peered":"2024-09-19T04:04:39.324402+0000","last_unstale":"2024-09-19T04:06:44.110927+0000","last_undegraded":"2024-09-19T04:06:44.110927+0000","last_fullsized":"2024-09-19T04:06:44.110927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:41.996135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.918796+0000","last_change":"2024-09-19T04:06:42.918320+0000","last_active":"2024-09-19T04:06:42.918796+0000","last_peered":"2024-09-19T04:06:42.918796+0000","last_clean":"2024-09-19T04:06:42.918796+0000","last_became_active":"2024-09-19T04:06:42.917819+0000","last_became_peered":"2024-09-19T04:06:42.917819+0000","last_unstale":"2024-09-19T04:06:42.918796+0000","last_undegraded":"2024-09-19T04:06:42.918796+0000","last_fullsized":"2024-09-19T04:06:42.918796+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:02.885774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,8],"acting":[2,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914539+0000","last_change":"2024-09-19T04:04:39.318556+0000","last_active":"2024-09-19T04:06:42.914539+0000","last_peered":"2024-09-19T04:06:42.914539+0000","last_clean":"2024-09-19T04:06:42.914539+0000","last_became_active":"2024-09-19T04:04:39.318200+0000","last_became_peered":"2024-09-19T04:04:39.318200+0000","last_unstale":"2024-09-19T04:06:42.914539+0000","last_undegraded":"2024-09-19T04:06:42.914539+0000","last_fullsized":"2024-09-19T04:06:42.914539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:40.847440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912503+0000","last_change":"2024-09-19T04:04:39.320698+0000","last_active":"2024-09-19T04:06:42.912503+0000","last_peered":"2024-09-19T04:06:42.912503+0000","last_clean":"2024-09-19T04:06:42.912503+0000","last_became_active":"2024-09-19T04:04:39.320419+0000","last_became_peered":"2024-09-19T04:04:39.320419+0000","last_unstale":"2024-09-19T04:06:42.912503+0000","last_undegraded":"2024-09-19T04:06:42.912503+0000","last_fullsized":"2024-09-19T04:06:42.912503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:32.708394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909421+0000","last_change":"2024-09-19T04:04:39.319886+0000","last_active":"2024-09-19T04:06:42.909421+0000","last_peered":"2024-09-19T04:06:42.909421+0000","last_clean":"2024-09-19T04:06:42.909421+0000","last_became_active":"2024-09-19T04:04:39.319217+0000","last_became_peered":"2024-09-19T04:04:39.319217+0000","last_unstale":"2024-09-19T04:06:42.909421+0000","last_undegraded":"2024-09-19T04:06:42.909421+0000","last_fullsized":"2024-09-19T04:06:42.909421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:38.829608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111374+0000","last_change":"2024-09-19T04:04:39.327639+0000","last_active":"2024-09-19T04:06:44.111374+0000","last_peered":"2024-09-19T04:06:44.111374+0000","last_clean":"2024-09-19T04:06:44.111374+0000","last_became_active":"2024-09-19T04:04:39.327349+0000","last_became_peered":"2024-09-19T04:04:39.327349+0000","last_unstale":"2024-09-19T04:06:44.111374+0000","last_undegraded":"2024-09-19T04:06:44.111374+0000","last_fullsized":"2024-09-19T04:06:44.111374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:31.056609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914540+0000","last_change":"2024-09-19T04:04:39.320253+0000","last_active":"2024-09-19T04:06:42.914540+0000","last_peered":"2024-09-19T04:06:42.914540+0000","last_clean":"2024-09-19T04:06:42.914540+0000","last_became_active":"2024-09-19T04:04:39.319851+0000","last_became_peered":"2024-09-19T04:04:39.319851+0000","last_unstale":"2024-09-19T04:06:42.914540+0000","last_undegraded":"2024-09-19T04:06:42.914540+0000","last_fullsized":"2024-09-19T04:06:42.914540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:53.631216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294843+0000","last_change":"2024-09-19T04:04:39.322779+0000","last_active":"2024-09-19T04:06:43.294843+0000","last_peered":"2024-09-19T04:06:43.294843+0000","last_clean":"2024-09-19T04:06:43.294843+0000","last_became_active":"2024-09-19T04:04:39.322498+0000","last_became_peered":"2024-09-19T04:04:39.322498+0000","last_unstale":"2024-09-19T04:06:43.294843+0000","last_undegraded":"2024-09-19T04:06:43.294843+0000","last_fullsized":"2024-09-19T04:06:43.294843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:24:11.956002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.911989+0000","last_change":"2024-09-19T04:04:39.322759+0000","last_active":"2024-09-19T04:06:42.911989+0000","last_peered":"2024-09-19T04:06:42.911989+0000","last_clean":"2024-09-19T04:06:42.911989+0000","last_became_active":"2024-09-19T04:04:39.322429+0000","last_became_peered":"2024-09-19T04:04:39.322429+0000","last_unstale":"2024-09-19T04:06:42.911989+0000","last_undegraded":"2024-09-19T04:06:42.911989+0000","last_fullsized":"2024-09-19T04:06:42.911989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:36.069931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294608+0000","last_change":"2024-09-19T04:04:39.320078+0000","last_active":"2024-09-19T04:06:43.294608+0000","last_peered":"2024-09-19T04:06:43.294608+0000","last_clean":"2024-09-19T04:06:43.294608+0000","last_became_active":"2024-09-19T04:04:39.319755+0000","last_became_peered":"2024-09-19T04:04:39.319755+0000","last_unstale":"2024-09-19T04:06:43.294608+0000","last_undegraded":"2024-09-19T04:06:43.294608+0000","last_fullsized":"2024-09-19T04:06:43.294608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:07.651712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111342+0000","last_change":"2024-09-19T04:04:39.319839+0000","last_active":"2024-09-19T04:06:44.111342+0000","last_peered":"2024-09-19T04:06:44.111342+0000","last_clean":"2024-09-19T04:06:44.111342+0000","last_became_active":"2024-09-19T04:04:39.319527+0000","last_became_peered":"2024-09-19T04:04:39.319527+0000","last_unstale":"2024-09-19T04:06:44.111342+0000","last_undegraded":"2024-09-19T04:06:44.111342+0000","last_fullsized":"2024-09-19T04:06:44.111342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:31.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909471+0000","last_change":"2024-09-19T04:04:39.315975+0000","last_active":"2024-09-19T04:06:42.909471+0000","last_peered":"2024-09-19T04:06:42.909471+0000","last_clean":"2024-09-19T04:06:42.909471+0000","last_became_active":"2024-09-19T04:04:39.315671+0000","last_became_peered":"2024-09-19T04:04:39.315671+0000","last_unstale":"2024-09-19T04:06:42.909471+0000","last_undegraded":"2024-09-19T04:06:42.909471+0000","last_fullsized":"2024-09-19T04:06:42.909471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:16.349853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111082+0000","last_change":"2024-09-19T04:04:39.316278+0000","last_active":"2024-09-19T04:06:44.111082+0000","last_peered":"2024-09-19T04:06:44.111082+0000","last_clean":"2024-09-19T04:06:44.111082+0000","last_became_active":"2024-09-19T04:04:39.315876+0000","last_became_peered":"2024-09-19T04:04:39.315876+0000","last_unstale":"2024-09-19T04:06:44.111082+0000","last_undegraded":"2024-09-19T04:06:44.111082+0000","last_fullsized":"2024-09-19T04:06:44.111082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:23.544405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914652+0000","last_change":"2024-09-19T04:04:39.318952+0000","last_active":"2024-09-19T04:06:42.914652+0000","last_peered":"2024-09-19T04:06:42.914652+0000","last_clean":"2024-09-19T04:06:42.914652+0000","last_became_active":"2024-09-19T04:04:39.318598+0000","last_became_peered":"2024-09-19T04:04:39.318598+0000","last_unstale":"2024-09-19T04:06:42.914652+0000","last_undegraded":"2024-09-19T04:06:42.914652+0000","last_fullsized":"2024-09-19T04:06:42.914652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:12.654611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913733+0000","last_change":"2024-09-19T04:04:39.311334+0000","last_active":"2024-09-19T04:06:42.913733+0000","last_peered":"2024-09-19T04:06:42.913733+0000","last_clean":"2024-09-19T04:06:42.913733+0000","last_became_active":"2024-09-19T04:04:39.311038+0000","last_became_peered":"2024-09-19T04:04:39.311038+0000","last_unstale":"2024-09-19T04:06:42.913733+0000","last_undegraded":"2024-09-19T04:06:42.913733+0000","last_fullsized":"2024-09-19T04:06:42.913733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:41.655363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111491+0000","last_change":"2024-09-19T04:04:39.319529+0000","last_active":"2024-09-19T04:06:44.111491+0000","last_peered":"2024-09-19T04:06:44.111491+0000","last_clean":"2024-09-19T04:06:44.111491+0000","last_became_active":"2024-09-19T04:04:39.318787+0000","last_became_peered":"2024-09-19T04:04:39.318787+0000","last_unstale":"2024-09-19T04:06:44.111491+0000","last_undegraded":"2024-09-19T04:06:44.111491+0000","last_fullsized":"2024-09-19T04:06:44.111491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.209422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110641+0000","last_change":"2024-09-19T04:04:39.320320+0000","last_active":"2024-09-19T04:06:44.110641+0000","last_peered":"2024-09-19T04:06:44.110641+0000","last_clean":"2024-09-19T04:06:44.110641+0000","last_became_active":"2024-09-19T04:04:39.319091+0000","last_became_peered":"2024-09-19T04:04:39.319091+0000","last_unstale":"2024-09-19T04:06:44.110641+0000","last_undegraded":"2024-09-19T04:06:44.110641+0000","last_fullsized":"2024-09-19T04:06:44.110641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:39.495079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912247+0000","last_change":"2024-09-19T04:04:39.323270+0000","last_active":"2024-09-19T04:06:42.912247+0000","last_peered":"2024-09-19T04:06:42.912247+0000","last_clean":"2024-09-19T04:06:42.912247+0000","last_became_active":"2024-09-19T04:04:39.322890+0000","last_became_peered":"2024-09-19T04:04:39.322890+0000","last_unstale":"2024-09-19T04:06:42.912247+0000","last_undegraded":"2024-09-19T04:06:42.912247+0000","last_fullsized":"2024-09-19T04:06:42.912247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:07.311654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912217+0000","last_change":"2024-09-19T04:04:39.317098+0000","last_active":"2024-09-19T04:06:42.912217+0000","last_peered":"2024-09-19T04:06:42.912217+0000","last_clean":"2024-09-19T04:06:42.912217+0000","last_became_active":"2024-09-19T04:04:39.316792+0000","last_became_peered":"2024-09-19T04:04:39.316792+0000","last_unstale":"2024-09-19T04:06:42.912217+0000","last_undegraded":"2024-09-19T04:06:42.912217+0000","last_fullsized":"2024-09-19T04:06:42.912217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.571149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909396+0000","last_change":"2024-09-19T04:04:39.326473+0000","last_active":"2024-09-19T04:06:42.909396+0000","last_peered":"2024-09-19T04:06:42.909396+0000","last_clean":"2024-09-19T04:06:42.909396+0000","last_became_active":"2024-09-19T04:04:39.325506+0000","last_became_peered":"2024-09-19T04:04:39.325506+0000","last_unstale":"2024-09-19T04:06:42.909396+0000","last_undegraded":"2024-09-19T04:06:42.909396+0000","last_fullsized":"2024-09-19T04:06:42.909396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:40.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915242+0000","last_change":"2024-09-19T04:04:39.323536+0000","last_active":"2024-09-19T04:06:42.915242+0000","last_peered":"2024-09-19T04:06:42.915242+0000","last_clean":"2024-09-19T04:06:42.915242+0000","last_became_active":"2024-09-19T04:04:39.323224+0000","last_became_peered":"2024-09-19T04:04:39.323224+0000","last_unstale":"2024-09-19T04:06:42.915242+0000","last_undegraded":"2024-09-19T04:06:42.915242+0000","last_fullsized":"2024-09-19T04:06:42.915242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:54.640599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419156+0000","last_change":"2024-09-19T04:04:39.323802+0000","last_active":"2024-09-19T04:06:43.419156+0000","last_peered":"2024-09-19T04:06:43.419156+0000","last_clean":"2024-09-19T04:06:43.419156+0000","last_became_active":"2024-09-19T04:04:39.323031+0000","last_became_peered":"2024-09-19T04:04:39.323031+0000","last_unstale":"2024-09-19T04:06:43.419156+0000","last_undegraded":"2024-09-19T04:06:43.419156+0000","last_fullsized":"2024-09-19T04:06:43.419156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:12.467182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914624+0000","last_change":"2024-09-19T04:04:39.324870+0000","last_active":"2024-09-19T04:06:42.914624+0000","last_peered":"2024-09-19T04:06:42.914624+0000","last_clean":"2024-09-19T04:06:42.914624+0000","last_became_active":"2024-09-19T04:04:39.324551+0000","last_became_peered":"2024-09-19T04:04:39.324551+0000","last_unstale":"2024-09-19T04:06:42.914624+0000","last_undegraded":"2024-09-19T04:06:42.914624+0000","last_fullsized":"2024-09-19T04:06:42.914624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:34.766437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111230+0000","last_change":"2024-09-19T04:04:39.329086+0000","last_active":"2024-09-19T04:06:44.111230+0000","last_peered":"2024-09-19T04:06:44.111230+0000","last_clean":"2024-09-19T04:06:44.111230+0000","last_became_active":"2024-09-19T04:04:39.328651+0000","last_became_peered":"2024-09-19T04:04:39.328651+0000","last_unstale":"2024-09-19T04:06:44.111230+0000","last_undegraded":"2024-09-19T04:06:44.111230+0000","last_fullsized":"2024-09-19T04:06:44.111230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:12.079162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294060+0000","last_change":"2024-09-19T04:04:39.325427+0000","last_active":"2024-09-19T04:06:43.294060+0000","last_peered":"2024-09-19T04:06:43.294060+0000","last_clean":"2024-09-19T04:06:43.294060+0000","last_became_active":"2024-09-19T04:04:39.325113+0000","last_became_peered":"2024-09-19T04:04:39.325113+0000","last_unstale":"2024-09-19T04:06:43.294060+0000","last_undegraded":"2024-09-19T04:06:43.294060+0000","last_fullsized":"2024-09-19T04:06:43.294060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:15.844503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418560+0000","last_change":"2024-09-19T04:04:39.325032+0000","last_active":"2024-09-19T04:06:43.418560+0000","last_peered":"2024-09-19T04:06:43.418560+0000","last_clean":"2024-09-19T04:06:43.418560+0000","last_became_active":"2024-09-19T04:04:39.324618+0000","last_became_peered":"2024-09-19T04:04:39.324618+0000","last_unstale":"2024-09-19T04:06:43.418560+0000","last_undegraded":"2024-09-19T04:06:43.418560+0000","last_fullsized":"2024-09-19T04:06:43.418560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:58.748613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913935+0000","last_change":"2024-09-19T04:04:39.318336+0000","last_active":"2024-09-19T04:06:42.913935+0000","last_peered":"2024-09-19T04:06:42.913935+0000","last_clean":"2024-09-19T04:06:42.913935+0000","last_became_active":"2024-09-19T04:04:39.317032+0000","last_became_peered":"2024-09-19T04:04:39.317032+0000","last_unstale":"2024-09-19T04:06:42.913935+0000","last_undegraded":"2024-09-19T04:06:42.913935+0000","last_fullsized":"2024-09-19T04:06:42.913935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:04.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916560+0000","last_change":"2024-09-19T04:04:39.321909+0000","last_active":"2024-09-19T04:06:42.916560+0000","last_peered":"2024-09-19T04:06:42.916560+0000","last_clean":"2024-09-19T04:06:42.916560+0000","last_became_active":"2024-09-19T04:04:39.320522+0000","last_became_peered":"2024-09-19T04:04:39.320522+0000","last_unstale":"2024-09-19T04:06:42.916560+0000","last_undegraded":"2024-09-19T04:06:42.916560+0000","last_fullsized":"2024-09-19T04:06:42.916560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:57.270998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915032+0000","last_change":"2024-09-19T04:04:39.324610+0000","last_active":"2024-09-19T04:06:42.915032+0000","last_peered":"2024-09-19T04:06:42.915032+0000","last_clean":"2024-09-19T04:06:42.915032+0000","last_became_active":"2024-09-19T04:04:39.324190+0000","last_became_peered":"2024-09-19T04:04:39.324190+0000","last_unstale":"2024-09-19T04:06:42.915032+0000","last_undegraded":"2024-09-19T04:06:42.915032+0000","last_fullsized":"2024-09-19T04:06:42.915032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:51.418210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912621+0000","last_change":"2024-09-19T04:04:39.324220+0000","last_active":"2024-09-19T04:06:42.912621+0000","last_peered":"2024-09-19T04:06:42.912621+0000","last_clean":"2024-09-19T04:06:42.912621+0000","last_became_active":"2024-09-19T04:04:39.323890+0000","last_became_peered":"2024-09-19T04:04:39.323890+0000","last_unstale":"2024-09-19T04:06:42.912621+0000","last_undegraded":"2024-09-19T04:06:42.912621+0000","last_fullsized":"2024-09-19T04:06:42.912621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:15.475088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912692+0000","last_change":"2024-09-19T04:04:39.317240+0000","last_active":"2024-09-19T04:06:42.912692+0000","last_peered":"2024-09-19T04:06:42.912692+0000","last_clean":"2024-09-19T04:06:42.912692+0000","last_became_active":"2024-09-19T04:04:39.316740+0000","last_became_peered":"2024-09-19T04:04:39.316740+0000","last_unstale":"2024-09-19T04:06:42.912692+0000","last_undegraded":"2024-09-19T04:06:42.912692+0000","last_fullsized":"2024-09-19T04:06:42.912692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:46.705467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111449+0000","last_change":"2024-09-19T04:04:39.319665+0000","last_active":"2024-09-19T04:06:44.111449+0000","last_peered":"2024-09-19T04:06:44.111449+0000","last_clean":"2024-09-19T04:06:44.111449+0000","last_became_active":"2024-09-19T04:04:39.319227+0000","last_became_peered":"2024-09-19T04:04:39.319227+0000","last_unstale":"2024-09-19T04:06:44.111449+0000","last_undegraded":"2024-09-19T04:06:44.111449+0000","last_fullsized":"2024-09-19T04:06:44.111449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:45.581254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69897,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1056,"num_read_kb":605,"num_write":257,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":60960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":217977,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248101,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481835,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1036288,"data_stored":529087,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710767,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1183744,"data_stored":1178206,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874293,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":533091,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":529351,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1105920,"data_stored":1126321,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528389,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":525033,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1121369,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":599492,"data_compressed":39195,"data_compressed_allocated":53248,"data_compressed_original":139264,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":524207,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":525749,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1101824,"data_stored":588388,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":525955,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":71619,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:07:07.722 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T04:07:07.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T04:07:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:08.254 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:07 smithi019 ceph-mon[30938]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T04:07:08.254 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:07 smithi019 ceph-mon[30938]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:08.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:07 smithi111 ceph-mon[35365]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T04:07:08.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:07 smithi111 ceph-mon[35365]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:07 smithi196 ceph-mon[35731]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T04:07:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:07 smithi196 ceph-mon[35731]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:08.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 -- 172.21.15.19:0/1546295911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c104e80 msgr2=0x7f0b7c105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 --2- 172.21.15.19:0/1546295911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c104e80 0x7f0b7c105330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009900 tx=0x7f0b7402a410 comp rx=0 tx=0).stop 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 -- 172.21.15.19:0/1546295911 shutdown_connections 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 --2- 172.21.15.19:0/1546295911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c104e80 0x7f0b7c105330 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 --2- 172.21.15.19:0/1546295911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b7c06a370 0x7f0b7c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 --2- 172.21.15.19:0/1546295911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b7c069f00 0x7f0b7c100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 -- 172.21.15.19:0/1546295911 >> 172.21.15.19:0/1546295911 conn(0x7f0b7c077210 msgr2=0x7f0b7c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 -- 172.21.15.19:0/1546295911 shutdown_connections 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.500+0000 7f0b832a4640 1 -- 172.21.15.19:0/1546295911 wait complete. 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b832a4640 1 Processor -- start 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b832a4640 1 -- start start 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b832a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c069f00 0x7f0b7c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b832a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b7c06a370 0x7f0b7c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b832a4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b7c104e80 0x7f0b7c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b832a4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c110eb0 con 0x7f0b7c069f00 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b832a4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c110d30 con 0x7f0b7c06a370 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b832a4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c111030 con 0x7f0b7c104e80 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b81040640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c069f00 0x7f0b7c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.501+0000 7f0b81040640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c069f00 0x7f0b7c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38970/0 (socket says 172.21.15.19:38970) 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81040640 1 -- 172.21.15.19:0/272519496 learned_addr learned my addr 172.21.15.19:0/272519496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b8083f640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b7c06a370 0x7f0b7c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81841640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b7c104e80 0x7f0b7c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81040640 1 -- 172.21.15.19:0/272519496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b7c104e80 msgr2=0x7f0b7c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81040640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b7c104e80 0x7f0b7c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81040640 1 -- 172.21.15.19:0/272519496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b7c06a370 msgr2=0x7f0b7c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81040640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b7c06a370 0x7f0b7c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81040640 1 -- 172.21.15.19:0/272519496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c19d3b0 con 0x7f0b7c069f00 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81841640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b7c104e80 0x7f0b7c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b81040640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c069f00 0x7f0b7c198400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800eab0 tx=0x7f0b7800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b6e7fc640 1 -- 172.21.15.19:0/272519496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800ce40 con 0x7f0b7c069f00 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b6e7fc640 1 -- 172.21.15.19:0/272519496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b78011950 con 0x7f0b7c069f00 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c19d6a0 con 0x7f0b7c069f00 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b6e7fc640 1 -- 172.21.15.19:0/272519496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78010ae0 con 0x7f0b7c069f00 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.502+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1a4f80 con 0x7f0b7c069f00 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.503+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c105330 con 0x7f0b7c069f00 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.505+0000 7f0b6e7fc640 1 -- 172.21.15.19:0/272519496 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f0b78011b00 con 0x7f0b7c069f00 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.505+0000 7f0b6e7fc640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0b680a32d0 0x7f0b680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.505+0000 7f0b8083f640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0b680a32d0 0x7f0b680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.505+0000 7f0b6e7fc640 1 -- 172.21.15.19:0/272519496 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f0b780d0500 con 0x7f0b7c069f00 2024-09-19T04:07:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.506+0000 7f0b8083f640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0b680a32d0 0x7f0b680a5790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0ffc30 tx=0x7f0b7000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.508+0000 7f0b6e7fc640 1 -- 172.21.15.19:0/272519496 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b78016030 con 0x7f0b7c069f00 2024-09-19T04:07:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.682+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 --> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b7c1098c0 con 0x7f0b680a32d0 2024-09-19T04:07:08.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.690+0000 7f0b6e7fc640 1 -- 172.21.15.19:0/272519496 <== mgr.14150 v2:172.21.15.19:6800/2387193681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340820 (secure 0 0 0) 0x7f0b7c1098c0 con 0x7f0b680a32d0 2024-09-19T04:07:08.694 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:08.696 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-19T04:07:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0b680a32d0 msgr2=0x7f0b680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0b680a32d0 0x7f0b680a5790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0ffc30 tx=0x7f0b7000b390 comp rx=0 tx=0).stop 2024-09-19T04:07:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c069f00 msgr2=0x7f0b7c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c069f00 0x7f0b7c198400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800eab0 tx=0x7f0b7800ef70 comp rx=0 tx=0).stop 2024-09-19T04:07:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 shutdown_connections 2024-09-19T04:07:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0b680a32d0 0x7f0b680a5790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b7c104e80 0x7f0b7c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b7c06a370 0x7f0b7c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 --2- 172.21.15.19:0/272519496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c069f00 0x7f0b7c198400 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:08.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 >> 172.21.15.19:0/272519496 conn(0x7f0b7c077210 msgr2=0x7f0b7c1091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:08.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 shutdown_connections 2024-09-19T04:07:08.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:08.694+0000 7f0b832a4640 1 -- 172.21.15.19:0/272519496 wait complete. 2024-09-19T04:07:09.370 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-19T04:07:08.445975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668785,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1215,"num_read_kb":744,"num_write":446,"num_write_kb":3099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1241696,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349416,"kb_used_data":12616,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592408,"statfs":{"total":1151940427776,"available":1151582625792,"internally_reserved":0,"allocated":12918784,"data_stored":8315151,"data_compressed":470438,"data_compressed_allocated":638976,"data_compressed_original":2809856,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8902,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":81,"num_write":10,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002964"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:06.746963+0000","last_change":"2024-09-19T04:04:37.214769+0000","last_active":"2024-09-19T04:07:06.746963+0000","last_peered":"2024-09-19T04:07:06.746963+0000","last_clean":"2024-09-19T04:07:06.746963+0000","last_became_active":"2024-09-19T04:04:37.214518+0000","last_became_peered":"2024-09-19T04:04:37.214518+0000","last_unstale":"2024-09-19T04:07:06.746963+0000","last_undegraded":"2024-09-19T04:07:06.746963+0000","last_fullsized":"2024-09-19T04:07:06.746963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:50.704566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294660+0000","last_change":"2024-09-19T04:04:39.329733+0000","last_active":"2024-09-19T04:06:43.294660+0000","last_peered":"2024-09-19T04:06:43.294660+0000","last_clean":"2024-09-19T04:06:43.294660+0000","last_became_active":"2024-09-19T04:04:39.329031+0000","last_became_peered":"2024-09-19T04:04:39.329031+0000","last_unstale":"2024-09-19T04:06:43.294660+0000","last_undegraded":"2024-09-19T04:06:43.294660+0000","last_fullsized":"2024-09-19T04:06:43.294660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:57.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913948+0000","last_change":"2024-09-19T04:04:46.362679+0000","last_active":"2024-09-19T04:06:42.913948+0000","last_peered":"2024-09-19T04:06:42.913948+0000","last_clean":"2024-09-19T04:06:42.913948+0000","last_became_active":"2024-09-19T04:04:46.362257+0000","last_became_peered":"2024-09-19T04:04:46.362257+0000","last_unstale":"2024-09-19T04:06:42.913948+0000","last_undegraded":"2024-09-19T04:06:42.913948+0000","last_fullsized":"2024-09-19T04:06:42.913948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.389839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294777+0000","last_change":"2024-09-19T04:04:37.647295+0000","last_active":"2024-09-19T04:06:43.294777+0000","last_peered":"2024-09-19T04:06:43.294777+0000","last_clean":"2024-09-19T04:06:43.294777+0000","last_became_active":"2024-09-19T04:04:37.646964+0000","last_became_peered":"2024-09-19T04:04:37.646964+0000","last_unstale":"2024-09-19T04:06:43.294777+0000","last_undegraded":"2024-09-19T04:06:43.294777+0000","last_fullsized":"2024-09-19T04:06:43.294777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:49.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913163+0000","last_change":"2024-09-19T04:04:39.318270+0000","last_active":"2024-09-19T04:06:42.913163+0000","last_peered":"2024-09-19T04:06:42.913163+0000","last_clean":"2024-09-19T04:06:42.913163+0000","last_became_active":"2024-09-19T04:04:39.317972+0000","last_became_peered":"2024-09-19T04:04:39.317972+0000","last_unstale":"2024-09-19T04:06:42.913163+0000","last_undegraded":"2024-09-19T04:06:42.913163+0000","last_fullsized":"2024-09-19T04:06:42.913163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:51:25.994112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914875+0000","last_change":"2024-09-19T04:04:46.355111+0000","last_active":"2024-09-19T04:06:42.914875+0000","last_peered":"2024-09-19T04:06:42.914875+0000","last_clean":"2024-09-19T04:06:42.914875+0000","last_became_active":"2024-09-19T04:04:46.354684+0000","last_became_peered":"2024-09-19T04:04:46.354684+0000","last_unstale":"2024-09-19T04:06:42.914875+0000","last_undegraded":"2024-09-19T04:06:42.914875+0000","last_fullsized":"2024-09-19T04:06:42.914875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:33.141690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917115+0000","last_change":"2024-09-19T04:04:37.211772+0000","last_active":"2024-09-19T04:06:42.917115+0000","last_peered":"2024-09-19T04:06:42.917115+0000","last_clean":"2024-09-19T04:06:42.917115+0000","last_became_active":"2024-09-19T04:04:37.211452+0000","last_became_peered":"2024-09-19T04:04:37.211452+0000","last_unstale":"2024-09-19T04:06:42.917115+0000","last_undegraded":"2024-09-19T04:06:42.917115+0000","last_fullsized":"2024-09-19T04:06:42.917115+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:57.781481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110814+0000","last_change":"2024-09-19T04:04:39.320791+0000","last_active":"2024-09-19T04:06:44.110814+0000","last_peered":"2024-09-19T04:06:44.110814+0000","last_clean":"2024-09-19T04:06:44.110814+0000","last_became_active":"2024-09-19T04:04:39.320206+0000","last_became_peered":"2024-09-19T04:04:39.320206+0000","last_unstale":"2024-09-19T04:06:44.110814+0000","last_undegraded":"2024-09-19T04:06:44.110814+0000","last_fullsized":"2024-09-19T04:06:44.110814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:17.385974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915320+0000","last_change":"2024-09-19T04:04:46.363375+0000","last_active":"2024-09-19T04:06:42.915320+0000","last_peered":"2024-09-19T04:06:42.915320+0000","last_clean":"2024-09-19T04:06:42.915320+0000","last_became_active":"2024-09-19T04:04:46.362800+0000","last_became_peered":"2024-09-19T04:04:46.362800+0000","last_unstale":"2024-09-19T04:06:42.915320+0000","last_undegraded":"2024-09-19T04:06:42.915320+0000","last_fullsized":"2024-09-19T04:06:42.915320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:55.651507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917083+0000","last_change":"2024-09-19T04:04:37.206218+0000","last_active":"2024-09-19T04:06:42.917083+0000","last_peered":"2024-09-19T04:06:42.917083+0000","last_clean":"2024-09-19T04:06:42.917083+0000","last_became_active":"2024-09-19T04:04:37.205675+0000","last_became_peered":"2024-09-19T04:04:37.205675+0000","last_unstale":"2024-09-19T04:06:42.917083+0000","last_undegraded":"2024-09-19T04:06:42.917083+0000","last_fullsized":"2024-09-19T04:06:42.917083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:27.664960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912735+0000","last_change":"2024-09-19T04:04:39.329667+0000","last_active":"2024-09-19T04:06:42.912735+0000","last_peered":"2024-09-19T04:06:42.912735+0000","last_clean":"2024-09-19T04:06:42.912735+0000","last_became_active":"2024-09-19T04:04:39.328180+0000","last_became_peered":"2024-09-19T04:04:39.328180+0000","last_unstale":"2024-09-19T04:06:42.912735+0000","last_undegraded":"2024-09-19T04:06:42.912735+0000","last_fullsized":"2024-09-19T04:06:42.912735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:59.782472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914994+0000","last_change":"2024-09-19T04:04:46.359432+0000","last_active":"2024-09-19T04:06:42.914994+0000","last_peered":"2024-09-19T04:06:42.914994+0000","last_clean":"2024-09-19T04:06:42.914994+0000","last_became_active":"2024-09-19T04:04:46.358966+0000","last_became_peered":"2024-09-19T04:04:46.358966+0000","last_unstale":"2024-09-19T04:06:42.914994+0000","last_undegraded":"2024-09-19T04:06:42.914994+0000","last_fullsized":"2024-09-19T04:06:42.914994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:41.537926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418761+0000","last_change":"2024-09-19T04:04:39.325084+0000","last_active":"2024-09-19T04:06:43.418761+0000","last_peered":"2024-09-19T04:06:43.418761+0000","last_clean":"2024-09-19T04:06:43.418761+0000","last_became_active":"2024-09-19T04:04:39.324126+0000","last_became_peered":"2024-09-19T04:04:39.324126+0000","last_unstale":"2024-09-19T04:06:43.418761+0000","last_undegraded":"2024-09-19T04:06:43.418761+0000","last_fullsized":"2024-09-19T04:06:43.418761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:13.523889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110894+0000","last_change":"2024-09-19T04:05:42.359098+0000","last_active":"2024-09-19T04:06:44.110894+0000","last_peered":"2024-09-19T04:06:44.110894+0000","last_clean":"2024-09-19T04:06:44.110894+0000","last_became_active":"2024-09-19T04:05:42.341384+0000","last_became_peered":"2024-09-19T04:05:42.341384+0000","last_unstale":"2024-09-19T04:06:44.110894+0000","last_undegraded":"2024-09-19T04:06:44.110894+0000","last_fullsized":"2024-09-19T04:06:44.110894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:15:57.418355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915058+0000","last_change":"2024-09-19T04:04:46.374921+0000","last_active":"2024-09-19T04:06:42.915058+0000","last_peered":"2024-09-19T04:06:42.915058+0000","last_clean":"2024-09-19T04:06:42.915058+0000","last_became_active":"2024-09-19T04:04:46.374466+0000","last_became_peered":"2024-09-19T04:04:46.374466+0000","last_unstale":"2024-09-19T04:06:42.915058+0000","last_undegraded":"2024-09-19T04:06:42.915058+0000","last_fullsized":"2024-09-19T04:06:42.915058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:46.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294259+0000","last_change":"2024-09-19T04:04:39.319948+0000","last_active":"2024-09-19T04:06:43.294259+0000","last_peered":"2024-09-19T04:06:43.294259+0000","last_clean":"2024-09-19T04:06:43.294259+0000","last_became_active":"2024-09-19T04:04:39.319675+0000","last_became_peered":"2024-09-19T04:04:39.319675+0000","last_unstale":"2024-09-19T04:06:43.294259+0000","last_undegraded":"2024-09-19T04:06:43.294259+0000","last_fullsized":"2024-09-19T04:06:43.294259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:08.818133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914433+0000","last_change":"2024-09-19T04:04:37.207705+0000","last_active":"2024-09-19T04:06:42.914433+0000","last_peered":"2024-09-19T04:06:42.914433+0000","last_clean":"2024-09-19T04:06:42.914433+0000","last_became_active":"2024-09-19T04:04:37.207251+0000","last_became_peered":"2024-09-19T04:04:37.207251+0000","last_unstale":"2024-09-19T04:06:42.914433+0000","last_undegraded":"2024-09-19T04:06:42.914433+0000","last_fullsized":"2024-09-19T04:06:42.914433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:20.835004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914955+0000","last_change":"2024-09-19T04:04:46.354463+0000","last_active":"2024-09-19T04:06:42.914955+0000","last_peered":"2024-09-19T04:06:42.914955+0000","last_clean":"2024-09-19T04:06:42.914955+0000","last_became_active":"2024-09-19T04:04:46.353754+0000","last_became_peered":"2024-09-19T04:04:46.353754+0000","last_unstale":"2024-09-19T04:06:42.914955+0000","last_undegraded":"2024-09-19T04:06:42.914955+0000","last_fullsized":"2024-09-19T04:06:42.914955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:09.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110995+0000","last_change":"2024-09-19T04:04:39.322727+0000","last_active":"2024-09-19T04:06:44.110995+0000","last_peered":"2024-09-19T04:06:44.110995+0000","last_clean":"2024-09-19T04:06:44.110995+0000","last_became_active":"2024-09-19T04:04:39.320989+0000","last_became_peered":"2024-09-19T04:04:39.320989+0000","last_unstale":"2024-09-19T04:06:44.110995+0000","last_undegraded":"2024-09-19T04:06:44.110995+0000","last_fullsized":"2024-09-19T04:06:44.110995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:06.536540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294398+0000","last_change":"2024-09-19T04:04:37.213602+0000","last_active":"2024-09-19T04:06:43.294398+0000","last_peered":"2024-09-19T04:06:43.294398+0000","last_clean":"2024-09-19T04:06:43.294398+0000","last_became_active":"2024-09-19T04:04:37.213240+0000","last_became_peered":"2024-09-19T04:04:37.213240+0000","last_unstale":"2024-09-19T04:06:43.294398+0000","last_undegraded":"2024-09-19T04:06:43.294398+0000","last_fullsized":"2024-09-19T04:06:43.294398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:09.822393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913644+0000","last_change":"2024-09-19T04:04:46.363921+0000","last_active":"2024-09-19T04:06:42.913644+0000","last_peered":"2024-09-19T04:06:42.913644+0000","last_clean":"2024-09-19T04:06:42.913644+0000","last_became_active":"2024-09-19T04:04:46.363476+0000","last_became_peered":"2024-09-19T04:04:46.363476+0000","last_unstale":"2024-09-19T04:06:42.913644+0000","last_undegraded":"2024-09-19T04:06:42.913644+0000","last_fullsized":"2024-09-19T04:06:42.913644+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:32.015782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917044+0000","last_change":"2024-09-19T04:04:39.316814+0000","last_active":"2024-09-19T04:06:42.917044+0000","last_peered":"2024-09-19T04:06:42.917044+0000","last_clean":"2024-09-19T04:06:42.917044+0000","last_became_active":"2024-09-19T04:04:39.316504+0000","last_became_peered":"2024-09-19T04:04:39.316504+0000","last_unstale":"2024-09-19T04:06:42.917044+0000","last_undegraded":"2024-09-19T04:06:42.917044+0000","last_fullsized":"2024-09-19T04:06:42.917044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110786+0000","last_change":"2024-09-19T04:04:37.647260+0000","last_active":"2024-09-19T04:06:44.110786+0000","last_peered":"2024-09-19T04:06:44.110786+0000","last_clean":"2024-09-19T04:06:44.110786+0000","last_became_active":"2024-09-19T04:04:37.646971+0000","last_became_peered":"2024-09-19T04:04:37.646971+0000","last_unstale":"2024-09-19T04:06:44.110786+0000","last_undegraded":"2024-09-19T04:06:44.110786+0000","last_fullsized":"2024-09-19T04:06:44.110786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:07.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915112+0000","last_change":"2024-09-19T04:05:42.342340+0000","last_active":"2024-09-19T04:06:42.915112+0000","last_peered":"2024-09-19T04:06:42.915112+0000","last_clean":"2024-09-19T04:06:42.915112+0000","last_became_active":"2024-09-19T04:05:42.341861+0000","last_became_peered":"2024-09-19T04:05:42.341861+0000","last_unstale":"2024-09-19T04:06:42.915112+0000","last_undegraded":"2024-09-19T04:06:42.915112+0000","last_fullsized":"2024-09-19T04:06:42.915112+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:27.520514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916892+0000","last_change":"2024-09-19T04:06:42.916356+0000","last_active":"2024-09-19T04:06:42.916892+0000","last_peered":"2024-09-19T04:06:42.916892+0000","last_clean":"2024-09-19T04:06:42.916892+0000","last_became_active":"2024-09-19T04:06:42.915926+0000","last_became_peered":"2024-09-19T04:06:42.915926+0000","last_unstale":"2024-09-19T04:06:42.916892+0000","last_undegraded":"2024-09-19T04:06:42.916892+0000","last_fullsized":"2024-09-19T04:06:42.916892+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:46.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915360+0000","last_change":"2024-09-19T04:04:37.703740+0000","last_active":"2024-09-19T04:06:42.915360+0000","last_peered":"2024-09-19T04:06:42.915360+0000","last_clean":"2024-09-19T04:06:42.915360+0000","last_became_active":"2024-09-19T04:04:37.703414+0000","last_became_peered":"2024-09-19T04:04:37.703414+0000","last_unstale":"2024-09-19T04:06:42.915360+0000","last_undegraded":"2024-09-19T04:06:42.915360+0000","last_fullsized":"2024-09-19T04:06:42.915360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:19.653559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914409+0000","last_change":"2024-09-19T04:04:46.354964+0000","last_active":"2024-09-19T04:06:42.914409+0000","last_peered":"2024-09-19T04:06:42.914409+0000","last_clean":"2024-09-19T04:06:42.914409+0000","last_became_active":"2024-09-19T04:04:46.354494+0000","last_became_peered":"2024-09-19T04:04:46.354494+0000","last_unstale":"2024-09-19T04:06:42.914409+0000","last_undegraded":"2024-09-19T04:06:42.914409+0000","last_fullsized":"2024-09-19T04:06:42.914409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:11.246134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294858+0000","last_change":"2024-09-19T04:04:39.315811+0000","last_active":"2024-09-19T04:06:43.294858+0000","last_peered":"2024-09-19T04:06:43.294858+0000","last_clean":"2024-09-19T04:06:43.294858+0000","last_became_active":"2024-09-19T04:04:39.315466+0000","last_became_peered":"2024-09-19T04:04:39.315466+0000","last_unstale":"2024-09-19T04:06:43.294858+0000","last_undegraded":"2024-09-19T04:06:43.294858+0000","last_fullsized":"2024-09-19T04:06:43.294858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:17.414765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'18","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:03.329381+0000","last_change":"2024-09-19T04:04:37.204206+0000","last_active":"2024-09-19T04:07:03.329381+0000","last_peered":"2024-09-19T04:07:03.329381+0000","last_clean":"2024-09-19T04:07:03.329381+0000","last_became_active":"2024-09-19T04:04:37.203845+0000","last_became_peered":"2024-09-19T04:04:37.203845+0000","last_unstale":"2024-09-19T04:07:03.329381+0000","last_undegraded":"2024-09-19T04:07:03.329381+0000","last_fullsized":"2024-09-19T04:07:03.329381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:52.868643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.293985+0000","last_change":"2024-09-19T04:04:46.375541+0000","last_active":"2024-09-19T04:06:43.293985+0000","last_peered":"2024-09-19T04:06:43.293985+0000","last_clean":"2024-09-19T04:06:43.293985+0000","last_became_active":"2024-09-19T04:04:46.375142+0000","last_became_peered":"2024-09-19T04:04:46.375142+0000","last_unstale":"2024-09-19T04:06:43.293985+0000","last_undegraded":"2024-09-19T04:06:43.293985+0000","last_fullsized":"2024-09-19T04:06:43.293985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.889459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909494+0000","last_change":"2024-09-19T04:04:39.324094+0000","last_active":"2024-09-19T04:06:42.909494+0000","last_peered":"2024-09-19T04:06:42.909494+0000","last_clean":"2024-09-19T04:06:42.909494+0000","last_became_active":"2024-09-19T04:04:39.322897+0000","last_became_peered":"2024-09-19T04:04:39.322897+0000","last_unstale":"2024-09-19T04:06:42.909494+0000","last_undegraded":"2024-09-19T04:06:42.909494+0000","last_fullsized":"2024-09-19T04:06:42.909494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:07.773110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.917125+0000","last_change":"2024-09-19T04:04:37.647686+0000","last_active":"2024-09-19T04:06:42.917125+0000","last_peered":"2024-09-19T04:06:42.917125+0000","last_clean":"2024-09-19T04:06:42.917125+0000","last_became_active":"2024-09-19T04:04:37.646979+0000","last_became_peered":"2024-09-19T04:04:37.646979+0000","last_unstale":"2024-09-19T04:06:42.917125+0000","last_undegraded":"2024-09-19T04:06:42.917125+0000","last_fullsized":"2024-09-19T04:06:42.917125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:48.063075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111034+0000","last_change":"2024-09-19T04:04:46.356078+0000","last_active":"2024-09-19T04:06:44.111034+0000","last_peered":"2024-09-19T04:06:44.111034+0000","last_clean":"2024-09-19T04:06:44.111034+0000","last_became_active":"2024-09-19T04:04:46.355630+0000","last_became_peered":"2024-09-19T04:04:46.355630+0000","last_unstale":"2024-09-19T04:06:44.111034+0000","last_undegraded":"2024-09-19T04:06:44.111034+0000","last_fullsized":"2024-09-19T04:06:44.111034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:45.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914196+0000","last_change":"2024-09-19T04:04:39.311067+0000","last_active":"2024-09-19T04:06:42.914196+0000","last_peered":"2024-09-19T04:06:42.914196+0000","last_clean":"2024-09-19T04:06:42.914196+0000","last_became_active":"2024-09-19T04:04:39.310725+0000","last_became_peered":"2024-09-19T04:04:39.310725+0000","last_unstale":"2024-09-19T04:06:42.914196+0000","last_undegraded":"2024-09-19T04:06:42.914196+0000","last_fullsized":"2024-09-19T04:06:42.914196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:47:11.488121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913434+0000","last_change":"2024-09-19T04:04:37.648335+0000","last_active":"2024-09-19T04:06:42.913434+0000","last_peered":"2024-09-19T04:06:42.913434+0000","last_clean":"2024-09-19T04:06:42.913434+0000","last_became_active":"2024-09-19T04:04:37.648000+0000","last_became_peered":"2024-09-19T04:04:37.648000+0000","last_unstale":"2024-09-19T04:06:42.913434+0000","last_undegraded":"2024-09-19T04:06:42.913434+0000","last_fullsized":"2024-09-19T04:06:42.913434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:16.102728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916512+0000","last_change":"2024-09-19T04:04:46.375653+0000","last_active":"2024-09-19T04:06:42.916512+0000","last_peered":"2024-09-19T04:06:42.916512+0000","last_clean":"2024-09-19T04:06:42.916512+0000","last_became_active":"2024-09-19T04:04:46.375132+0000","last_became_peered":"2024-09-19T04:04:46.375132+0000","last_unstale":"2024-09-19T04:06:42.916512+0000","last_undegraded":"2024-09-19T04:06:42.916512+0000","last_fullsized":"2024-09-19T04:06:42.916512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:03.332697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914089+0000","last_change":"2024-09-19T04:04:39.318155+0000","last_active":"2024-09-19T04:06:42.914089+0000","last_peered":"2024-09-19T04:06:42.914089+0000","last_clean":"2024-09-19T04:06:42.914089+0000","last_became_active":"2024-09-19T04:04:39.316710+0000","last_became_peered":"2024-09-19T04:04:39.316710+0000","last_unstale":"2024-09-19T04:06:42.914089+0000","last_undegraded":"2024-09-19T04:06:42.914089+0000","last_fullsized":"2024-09-19T04:06:42.914089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:18:59.084523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:03.743187+0000","last_change":"2024-09-19T04:04:37.210695+0000","last_active":"2024-09-19T04:07:03.743187+0000","last_peered":"2024-09-19T04:07:03.743187+0000","last_clean":"2024-09-19T04:07:03.743187+0000","last_became_active":"2024-09-19T04:04:37.210334+0000","last_became_peered":"2024-09-19T04:04:37.210334+0000","last_unstale":"2024-09-19T04:07:03.743187+0000","last_undegraded":"2024-09-19T04:07:03.743187+0000","last_fullsized":"2024-09-19T04:07:03.743187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:12.545248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909313+0000","last_change":"2024-09-19T04:04:46.358130+0000","last_active":"2024-09-19T04:06:42.909313+0000","last_peered":"2024-09-19T04:06:42.909313+0000","last_clean":"2024-09-19T04:06:42.909313+0000","last_became_active":"2024-09-19T04:04:46.356810+0000","last_became_peered":"2024-09-19T04:04:46.356810+0000","last_unstale":"2024-09-19T04:06:42.909313+0000","last_undegraded":"2024-09-19T04:06:42.909313+0000","last_fullsized":"2024-09-19T04:06:42.909313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:49.884478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916817+0000","last_change":"2024-09-19T04:06:42.916354+0000","last_active":"2024-09-19T04:06:42.916817+0000","last_peered":"2024-09-19T04:06:42.916817+0000","last_clean":"2024-09-19T04:06:42.916817+0000","last_became_active":"2024-09-19T04:06:42.915943+0000","last_became_peered":"2024-09-19T04:06:42.915943+0000","last_unstale":"2024-09-19T04:06:42.916817+0000","last_undegraded":"2024-09-19T04:06:42.916817+0000","last_fullsized":"2024-09-19T04:06:42.916817+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:24.289049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:04.744026+0000","last_change":"2024-09-19T04:04:37.647310+0000","last_active":"2024-09-19T04:07:04.744026+0000","last_peered":"2024-09-19T04:07:04.744026+0000","last_clean":"2024-09-19T04:07:04.744026+0000","last_became_active":"2024-09-19T04:04:37.646903+0000","last_became_peered":"2024-09-19T04:04:37.646903+0000","last_unstale":"2024-09-19T04:07:04.744026+0000","last_undegraded":"2024-09-19T04:07:04.744026+0000","last_fullsized":"2024-09-19T04:07:04.744026+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:20.097472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909319+0000","last_change":"2024-09-19T04:04:46.375261+0000","last_active":"2024-09-19T04:06:42.909319+0000","last_peered":"2024-09-19T04:06:42.909319+0000","last_clean":"2024-09-19T04:06:42.909319+0000","last_became_active":"2024-09-19T04:04:46.374489+0000","last_became_peered":"2024-09-19T04:04:46.374489+0000","last_unstale":"2024-09-19T04:06:42.909319+0000","last_undegraded":"2024-09-19T04:06:42.909319+0000","last_fullsized":"2024-09-19T04:06:42.909319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:05.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.908952+0000","last_change":"2024-09-19T04:04:39.322265+0000","last_active":"2024-09-19T04:06:42.908952+0000","last_peered":"2024-09-19T04:06:42.908952+0000","last_clean":"2024-09-19T04:06:42.908952+0000","last_became_active":"2024-09-19T04:04:39.321938+0000","last_became_peered":"2024-09-19T04:04:39.321938+0000","last_unstale":"2024-09-19T04:06:42.908952+0000","last_undegraded":"2024-09-19T04:06:42.908952+0000","last_fullsized":"2024-09-19T04:06:42.908952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:19.220661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418797+0000","last_change":"2024-09-19T04:04:37.211330+0000","last_active":"2024-09-19T04:06:43.418797+0000","last_peered":"2024-09-19T04:06:43.418797+0000","last_clean":"2024-09-19T04:06:43.418797+0000","last_became_active":"2024-09-19T04:04:37.210788+0000","last_became_peered":"2024-09-19T04:04:37.210788+0000","last_unstale":"2024-09-19T04:06:43.418797+0000","last_undegraded":"2024-09-19T04:06:43.418797+0000","last_fullsized":"2024-09-19T04:06:43.418797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:43.169897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913106+0000","last_change":"2024-09-19T04:05:42.465969+0000","last_active":"2024-09-19T04:06:42.913106+0000","last_peered":"2024-09-19T04:06:42.913106+0000","last_clean":"2024-09-19T04:06:42.913106+0000","last_became_active":"2024-09-19T04:05:42.465621+0000","last_became_peered":"2024-09-19T04:05:42.465621+0000","last_unstale":"2024-09-19T04:06:42.913106+0000","last_undegraded":"2024-09-19T04:06:42.913106+0000","last_fullsized":"2024-09-19T04:06:42.913106+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:29.802141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,10],"acting":[5,1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294450+0000","last_change":"2024-09-19T04:04:39.329480+0000","last_active":"2024-09-19T04:06:43.294450+0000","last_peered":"2024-09-19T04:06:43.294450+0000","last_clean":"2024-09-19T04:06:43.294450+0000","last_became_active":"2024-09-19T04:04:39.329008+0000","last_became_peered":"2024-09-19T04:04:39.329008+0000","last_unstale":"2024-09-19T04:06:43.294450+0000","last_undegraded":"2024-09-19T04:06:43.294450+0000","last_fullsized":"2024-09-19T04:06:43.294450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:06.782579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915484+0000","last_change":"2024-09-19T04:04:37.216224+0000","last_active":"2024-09-19T04:06:42.915484+0000","last_peered":"2024-09-19T04:06:42.915484+0000","last_clean":"2024-09-19T04:06:42.915484+0000","last_became_active":"2024-09-19T04:04:37.215846+0000","last_became_peered":"2024-09-19T04:04:37.215846+0000","last_unstale":"2024-09-19T04:06:42.915484+0000","last_undegraded":"2024-09-19T04:06:42.915484+0000","last_fullsized":"2024-09-19T04:06:42.915484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:57.829694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418966+0000","last_change":"2024-09-19T04:03:25.869634+0000","last_active":"2024-09-19T04:06:43.418966+0000","last_peered":"2024-09-19T04:06:43.418966+0000","last_clean":"2024-09-19T04:06:43.418966+0000","last_became_active":"2024-09-19T04:03:25.869250+0000","last_became_peered":"2024-09-19T04:03:25.869250+0000","last_unstale":"2024-09-19T04:06:43.418966+0000","last_undegraded":"2024-09-19T04:06:43.418966+0000","last_fullsized":"2024-09-19T04:06:43.418966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T03:59:59.253417+0000","last_clean_scrub_stamp":"2024-09-19T03:59:59.253417+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:51.956932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419061+0000","last_change":"2024-09-19T04:04:46.357123+0000","last_active":"2024-09-19T04:06:43.419061+0000","last_peered":"2024-09-19T04:06:43.419061+0000","last_clean":"2024-09-19T04:06:43.419061+0000","last_became_active":"2024-09-19T04:04:46.356671+0000","last_became_peered":"2024-09-19T04:04:46.356671+0000","last_unstale":"2024-09-19T04:06:43.419061+0000","last_undegraded":"2024-09-19T04:06:43.419061+0000","last_fullsized":"2024-09-19T04:06:43.419061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:09.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419189+0000","last_change":"2024-09-19T04:04:39.322487+0000","last_active":"2024-09-19T04:06:43.419189+0000","last_peered":"2024-09-19T04:06:43.419189+0000","last_clean":"2024-09-19T04:06:43.419189+0000","last_became_active":"2024-09-19T04:04:39.321994+0000","last_became_peered":"2024-09-19T04:04:39.321994+0000","last_unstale":"2024-09-19T04:06:43.419189+0000","last_undegraded":"2024-09-19T04:06:43.419189+0000","last_fullsized":"2024-09-19T04:06:43.419189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:56.977405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:06.759172+0000","last_change":"2024-09-19T04:04:37.703452+0000","last_active":"2024-09-19T04:07:06.759172+0000","last_peered":"2024-09-19T04:07:06.759172+0000","last_clean":"2024-09-19T04:07:06.759172+0000","last_became_active":"2024-09-19T04:04:37.703048+0000","last_became_peered":"2024-09-19T04:04:37.703048+0000","last_unstale":"2024-09-19T04:07:06.759172+0000","last_undegraded":"2024-09-19T04:07:06.759172+0000","last_fullsized":"2024-09-19T04:07:06.759172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:16:29.051617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110726+0000","last_change":"2024-09-19T04:04:46.361229+0000","last_active":"2024-09-19T04:06:44.110726+0000","last_peered":"2024-09-19T04:06:44.110726+0000","last_clean":"2024-09-19T04:06:44.110726+0000","last_became_active":"2024-09-19T04:04:46.360729+0000","last_became_peered":"2024-09-19T04:04:46.360729+0000","last_unstale":"2024-09-19T04:06:44.110726+0000","last_undegraded":"2024-09-19T04:06:44.110726+0000","last_fullsized":"2024-09-19T04:06:44.110726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:48.022778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418940+0000","last_change":"2024-09-19T04:04:39.328598+0000","last_active":"2024-09-19T04:06:43.418940+0000","last_peered":"2024-09-19T04:06:43.418940+0000","last_clean":"2024-09-19T04:06:43.418940+0000","last_became_active":"2024-09-19T04:04:39.328247+0000","last_became_peered":"2024-09-19T04:04:39.328247+0000","last_unstale":"2024-09-19T04:06:43.418940+0000","last_undegraded":"2024-09-19T04:06:43.418940+0000","last_fullsized":"2024-09-19T04:06:43.418940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:34.916487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913045+0000","last_change":"2024-09-19T04:04:37.212065+0000","last_active":"2024-09-19T04:06:42.913045+0000","last_peered":"2024-09-19T04:06:42.913045+0000","last_clean":"2024-09-19T04:06:42.913045+0000","last_became_active":"2024-09-19T04:04:37.211520+0000","last_became_peered":"2024-09-19T04:04:37.211520+0000","last_unstale":"2024-09-19T04:06:42.913045+0000","last_undegraded":"2024-09-19T04:06:42.913045+0000","last_fullsized":"2024-09-19T04:06:42.913045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:53.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912491+0000","last_change":"2024-09-19T04:04:46.358558+0000","last_active":"2024-09-19T04:06:42.912491+0000","last_peered":"2024-09-19T04:06:42.912491+0000","last_clean":"2024-09-19T04:06:42.912491+0000","last_became_active":"2024-09-19T04:04:46.358158+0000","last_became_peered":"2024-09-19T04:04:46.358158+0000","last_unstale":"2024-09-19T04:06:42.912491+0000","last_undegraded":"2024-09-19T04:06:42.912491+0000","last_fullsized":"2024-09-19T04:06:42.912491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:16.627051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419310+0000","last_change":"2024-09-19T04:04:39.325187+0000","last_active":"2024-09-19T04:06:43.419310+0000","last_peered":"2024-09-19T04:06:43.419310+0000","last_clean":"2024-09-19T04:06:43.419310+0000","last_became_active":"2024-09-19T04:04:39.324644+0000","last_became_peered":"2024-09-19T04:04:39.324644+0000","last_unstale":"2024-09-19T04:06:43.419310+0000","last_undegraded":"2024-09-19T04:06:43.419310+0000","last_fullsized":"2024-09-19T04:06:43.419310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:11.879273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909362+0000","last_change":"2024-09-19T04:04:37.205389+0000","last_active":"2024-09-19T04:06:42.909362+0000","last_peered":"2024-09-19T04:06:42.909362+0000","last_clean":"2024-09-19T04:06:42.909362+0000","last_became_active":"2024-09-19T04:04:37.205117+0000","last_became_peered":"2024-09-19T04:04:37.205117+0000","last_unstale":"2024-09-19T04:06:42.909362+0000","last_undegraded":"2024-09-19T04:06:42.909362+0000","last_fullsized":"2024-09-19T04:06:42.909362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:49.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916378+0000","last_change":"2024-09-19T04:05:42.342246+0000","last_active":"2024-09-19T04:06:42.916378+0000","last_peered":"2024-09-19T04:06:42.916378+0000","last_clean":"2024-09-19T04:06:42.916378+0000","last_became_active":"2024-09-19T04:05:42.341628+0000","last_became_peered":"2024-09-19T04:05:42.341628+0000","last_unstale":"2024-09-19T04:06:42.916378+0000","last_undegraded":"2024-09-19T04:06:42.916378+0000","last_fullsized":"2024-09-19T04:06:42.916378+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:16.637647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111363+0000","last_change":"2024-09-19T04:04:39.328841+0000","last_active":"2024-09-19T04:06:44.111363+0000","last_peered":"2024-09-19T04:06:44.111363+0000","last_clean":"2024-09-19T04:06:44.111363+0000","last_became_active":"2024-09-19T04:04:39.328470+0000","last_became_peered":"2024-09-19T04:04:39.328470+0000","last_unstale":"2024-09-19T04:06:44.111363+0000","last_undegraded":"2024-09-19T04:06:44.111363+0000","last_fullsized":"2024-09-19T04:06:44.111363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:17.701469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111344+0000","last_change":"2024-09-19T04:05:42.353456+0000","last_active":"2024-09-19T04:06:44.111344+0000","last_peered":"2024-09-19T04:06:44.111344+0000","last_clean":"2024-09-19T04:06:44.111344+0000","last_became_active":"2024-09-19T04:05:42.340710+0000","last_became_peered":"2024-09-19T04:05:42.340710+0000","last_unstale":"2024-09-19T04:06:44.111344+0000","last_undegraded":"2024-09-19T04:06:44.111344+0000","last_fullsized":"2024-09-19T04:06:44.111344+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:45.262693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912021+0000","last_change":"2024-09-19T04:04:46.361199+0000","last_active":"2024-09-19T04:06:42.912021+0000","last_peered":"2024-09-19T04:06:42.912021+0000","last_clean":"2024-09-19T04:06:42.912021+0000","last_became_active":"2024-09-19T04:04:46.360639+0000","last_became_peered":"2024-09-19T04:04:46.360639+0000","last_unstale":"2024-09-19T04:06:42.912021+0000","last_undegraded":"2024-09-19T04:06:42.912021+0000","last_fullsized":"2024-09-19T04:06:42.912021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:50.185760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912115+0000","last_change":"2024-09-19T04:04:39.328529+0000","last_active":"2024-09-19T04:06:42.912115+0000","last_peered":"2024-09-19T04:06:42.912115+0000","last_clean":"2024-09-19T04:06:42.912115+0000","last_became_active":"2024-09-19T04:04:39.328174+0000","last_became_peered":"2024-09-19T04:04:39.328174+0000","last_unstale":"2024-09-19T04:06:42.912115+0000","last_undegraded":"2024-09-19T04:06:42.912115+0000","last_fullsized":"2024-09-19T04:06:42.912115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:02.503672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916702+0000","last_change":"2024-09-19T04:04:37.647919+0000","last_active":"2024-09-19T04:06:42.916702+0000","last_peered":"2024-09-19T04:06:42.916702+0000","last_clean":"2024-09-19T04:06:42.916702+0000","last_became_active":"2024-09-19T04:04:37.647305+0000","last_became_peered":"2024-09-19T04:04:37.647305+0000","last_unstale":"2024-09-19T04:06:42.916702+0000","last_undegraded":"2024-09-19T04:06:42.916702+0000","last_fullsized":"2024-09-19T04:06:42.916702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:55:07.035465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909285+0000","last_change":"2024-09-19T04:04:46.360362+0000","last_active":"2024-09-19T04:06:42.909285+0000","last_peered":"2024-09-19T04:06:42.909285+0000","last_clean":"2024-09-19T04:06:42.909285+0000","last_became_active":"2024-09-19T04:04:46.359470+0000","last_became_peered":"2024-09-19T04:04:46.359470+0000","last_unstale":"2024-09-19T04:06:42.909285+0000","last_undegraded":"2024-09-19T04:06:42.909285+0000","last_fullsized":"2024-09-19T04:06:42.909285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:43.804073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418669+0000","last_change":"2024-09-19T04:04:39.326090+0000","last_active":"2024-09-19T04:06:43.418669+0000","last_peered":"2024-09-19T04:06:43.418669+0000","last_clean":"2024-09-19T04:06:43.418669+0000","last_became_active":"2024-09-19T04:04:39.325751+0000","last_became_peered":"2024-09-19T04:04:39.325751+0000","last_unstale":"2024-09-19T04:06:43.418669+0000","last_undegraded":"2024-09-19T04:06:43.418669+0000","last_fullsized":"2024-09-19T04:06:43.418669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:25.357467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:03.743096+0000","last_change":"2024-09-19T04:04:37.215107+0000","last_active":"2024-09-19T04:07:03.743096+0000","last_peered":"2024-09-19T04:07:03.743096+0000","last_clean":"2024-09-19T04:07:03.743096+0000","last_became_active":"2024-09-19T04:04:37.214693+0000","last_became_peered":"2024-09-19T04:04:37.214693+0000","last_unstale":"2024-09-19T04:07:03.743096+0000","last_undegraded":"2024-09-19T04:07:03.743096+0000","last_fullsized":"2024-09-19T04:07:03.743096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:56.535316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913700+0000","last_change":"2024-09-19T04:04:46.363625+0000","last_active":"2024-09-19T04:06:42.913700+0000","last_peered":"2024-09-19T04:06:42.913700+0000","last_clean":"2024-09-19T04:06:42.913700+0000","last_became_active":"2024-09-19T04:04:46.363207+0000","last_became_peered":"2024-09-19T04:04:46.363207+0000","last_unstale":"2024-09-19T04:06:42.913700+0000","last_undegraded":"2024-09-19T04:06:42.913700+0000","last_fullsized":"2024-09-19T04:06:42.913700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:30.251168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418748+0000","last_change":"2024-09-19T04:05:42.466411+0000","last_active":"2024-09-19T04:06:43.418748+0000","last_peered":"2024-09-19T04:06:43.418748+0000","last_clean":"2024-09-19T04:06:43.418748+0000","last_became_active":"2024-09-19T04:05:42.465999+0000","last_became_peered":"2024-09-19T04:05:42.465999+0000","last_unstale":"2024-09-19T04:06:43.418748+0000","last_undegraded":"2024-09-19T04:06:43.418748+0000","last_fullsized":"2024-09-19T04:06:43.418748+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:51.289726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912476+0000","last_change":"2024-09-19T04:04:37.212103+0000","last_active":"2024-09-19T04:06:42.912476+0000","last_peered":"2024-09-19T04:06:42.912476+0000","last_clean":"2024-09-19T04:06:42.912476+0000","last_became_active":"2024-09-19T04:04:37.211739+0000","last_became_peered":"2024-09-19T04:04:37.211739+0000","last_unstale":"2024-09-19T04:06:42.912476+0000","last_undegraded":"2024-09-19T04:06:42.912476+0000","last_fullsized":"2024-09-19T04:06:42.912476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:06.588194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110915+0000","last_change":"2024-09-19T04:04:46.361975+0000","last_active":"2024-09-19T04:06:44.110915+0000","last_peered":"2024-09-19T04:06:44.110915+0000","last_clean":"2024-09-19T04:06:44.110915+0000","last_became_active":"2024-09-19T04:04:46.361492+0000","last_became_peered":"2024-09-19T04:04:46.361492+0000","last_unstale":"2024-09-19T04:06:44.110915+0000","last_undegraded":"2024-09-19T04:06:44.110915+0000","last_fullsized":"2024-09-19T04:06:44.110915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:52.723836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909144+0000","last_change":"2024-09-19T04:04:39.317756+0000","last_active":"2024-09-19T04:06:42.909144+0000","last_peered":"2024-09-19T04:06:42.909144+0000","last_clean":"2024-09-19T04:06:42.909144+0000","last_became_active":"2024-09-19T04:04:39.317322+0000","last_became_peered":"2024-09-19T04:04:39.317322+0000","last_unstale":"2024-09-19T04:06:42.909144+0000","last_undegraded":"2024-09-19T04:06:42.909144+0000","last_fullsized":"2024-09-19T04:06:42.909144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:26.291574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914459+0000","last_change":"2024-09-19T04:04:37.213892+0000","last_active":"2024-09-19T04:06:42.914459+0000","last_peered":"2024-09-19T04:06:42.914459+0000","last_clean":"2024-09-19T04:06:42.914459+0000","last_became_active":"2024-09-19T04:04:37.213350+0000","last_became_peered":"2024-09-19T04:04:37.213350+0000","last_unstale":"2024-09-19T04:06:42.914459+0000","last_undegraded":"2024-09-19T04:06:42.914459+0000","last_fullsized":"2024-09-19T04:06:42.914459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:40.065717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912727+0000","last_change":"2024-09-19T04:04:46.360566+0000","last_active":"2024-09-19T04:06:42.912727+0000","last_peered":"2024-09-19T04:06:42.912727+0000","last_clean":"2024-09-19T04:06:42.912727+0000","last_became_active":"2024-09-19T04:04:46.360143+0000","last_became_peered":"2024-09-19T04:04:46.360143+0000","last_unstale":"2024-09-19T04:06:42.912727+0000","last_undegraded":"2024-09-19T04:06:42.912727+0000","last_fullsized":"2024-09-19T04:06:42.912727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:59.997088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916958+0000","last_change":"2024-09-19T04:04:39.318363+0000","last_active":"2024-09-19T04:06:42.916958+0000","last_peered":"2024-09-19T04:06:42.916958+0000","last_clean":"2024-09-19T04:06:42.916958+0000","last_became_active":"2024-09-19T04:04:39.318003+0000","last_became_peered":"2024-09-19T04:04:39.318003+0000","last_unstale":"2024-09-19T04:06:42.916958+0000","last_undegraded":"2024-09-19T04:06:42.916958+0000","last_fullsized":"2024-09-19T04:06:42.916958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:54.128795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'41","reported_seq":323,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:07.467439+0000","last_change":"2024-09-19T04:05:42.349944+0000","last_active":"2024-09-19T04:07:07.467439+0000","last_peered":"2024-09-19T04:07:07.467439+0000","last_clean":"2024-09-19T04:07:07.467439+0000","last_became_active":"2024-09-19T04:05:42.339605+0000","last_became_peered":"2024-09-19T04:05:42.339605+0000","last_unstale":"2024-09-19T04:07:07.467439+0000","last_undegraded":"2024-09-19T04:07:07.467439+0000","last_fullsized":"2024-09-19T04:07:07.467439+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:17.508316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70919,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":223,"num_read_kb":183,"num_write":43,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":60938,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912932+0000","last_change":"2024-09-19T04:04:46.362967+0000","last_active":"2024-09-19T04:06:42.912932+0000","last_peered":"2024-09-19T04:06:42.912932+0000","last_clean":"2024-09-19T04:06:42.912932+0000","last_became_active":"2024-09-19T04:04:46.362529+0000","last_became_peered":"2024-09-19T04:04:46.362529+0000","last_unstale":"2024-09-19T04:06:42.912932+0000","last_undegraded":"2024-09-19T04:06:42.912932+0000","last_fullsized":"2024-09-19T04:06:42.912932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:01.821697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110944+0000","last_change":"2024-09-19T04:04:39.323499+0000","last_active":"2024-09-19T04:06:44.110944+0000","last_peered":"2024-09-19T04:06:44.110944+0000","last_clean":"2024-09-19T04:06:44.110944+0000","last_became_active":"2024-09-19T04:04:39.323076+0000","last_became_peered":"2024-09-19T04:04:39.323076+0000","last_unstale":"2024-09-19T04:06:44.110944+0000","last_undegraded":"2024-09-19T04:06:44.110944+0000","last_fullsized":"2024-09-19T04:06:44.110944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:10.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913160+0000","last_change":"2024-09-19T04:04:37.647927+0000","last_active":"2024-09-19T04:06:42.913160+0000","last_peered":"2024-09-19T04:06:42.913160+0000","last_clean":"2024-09-19T04:06:42.913160+0000","last_became_active":"2024-09-19T04:04:37.647279+0000","last_became_peered":"2024-09-19T04:04:37.647279+0000","last_unstale":"2024-09-19T04:06:42.913160+0000","last_undegraded":"2024-09-19T04:06:42.913160+0000","last_fullsized":"2024-09-19T04:06:42.913160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:05:50.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914938+0000","last_change":"2024-09-19T04:04:46.363562+0000","last_active":"2024-09-19T04:06:42.914938+0000","last_peered":"2024-09-19T04:06:42.914938+0000","last_clean":"2024-09-19T04:06:42.914938+0000","last_became_active":"2024-09-19T04:04:46.362915+0000","last_became_peered":"2024-09-19T04:04:46.362915+0000","last_unstale":"2024-09-19T04:06:42.914938+0000","last_undegraded":"2024-09-19T04:06:42.914938+0000","last_fullsized":"2024-09-19T04:06:42.914938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:08.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.420917+0000","last_change":"2024-09-19T04:06:43.420595+0000","last_active":"2024-09-19T04:06:43.420917+0000","last_peered":"2024-09-19T04:06:43.420917+0000","last_clean":"2024-09-19T04:06:43.420917+0000","last_became_active":"2024-09-19T04:06:43.420316+0000","last_became_peered":"2024-09-19T04:06:43.420316+0000","last_unstale":"2024-09-19T04:06:43.420917+0000","last_undegraded":"2024-09-19T04:06:43.420917+0000","last_fullsized":"2024-09-19T04:06:43.420917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:12:54.798028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419119+0000","last_change":"2024-09-19T04:04:37.212393+0000","last_active":"2024-09-19T04:06:43.419119+0000","last_peered":"2024-09-19T04:06:43.419119+0000","last_clean":"2024-09-19T04:06:43.419119+0000","last_became_active":"2024-09-19T04:04:37.212013+0000","last_became_peered":"2024-09-19T04:04:37.212013+0000","last_unstale":"2024-09-19T04:06:43.419119+0000","last_undegraded":"2024-09-19T04:06:43.419119+0000","last_fullsized":"2024-09-19T04:06:43.419119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:37.981075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916368+0000","last_change":"2024-09-19T04:05:43.151378+0000","last_active":"2024-09-19T04:06:42.916368+0000","last_peered":"2024-09-19T04:06:42.916368+0000","last_clean":"2024-09-19T04:06:42.916368+0000","last_became_active":"2024-09-19T04:05:43.150907+0000","last_became_peered":"2024-09-19T04:05:43.150907+0000","last_unstale":"2024-09-19T04:06:42.916368+0000","last_undegraded":"2024-09-19T04:06:42.916368+0000","last_fullsized":"2024-09-19T04:06:42.916368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:09.542371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111185+0000","last_change":"2024-09-19T04:04:39.319705+0000","last_active":"2024-09-19T04:06:44.111185+0000","last_peered":"2024-09-19T04:06:44.111185+0000","last_clean":"2024-09-19T04:06:44.111185+0000","last_became_active":"2024-09-19T04:04:39.318065+0000","last_became_peered":"2024-09-19T04:04:39.318065+0000","last_unstale":"2024-09-19T04:06:44.111185+0000","last_undegraded":"2024-09-19T04:06:44.111185+0000","last_fullsized":"2024-09-19T04:06:44.111185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:15.630927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":256,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:07.760198+0000","last_change":"2024-09-19T04:04:37.215192+0000","last_active":"2024-09-19T04:07:07.760198+0000","last_peered":"2024-09-19T04:07:07.760198+0000","last_clean":"2024-09-19T04:07:07.760198+0000","last_became_active":"2024-09-19T04:04:37.214872+0000","last_became_peered":"2024-09-19T04:04:37.214872+0000","last_unstale":"2024-09-19T04:07:07.760198+0000","last_undegraded":"2024-09-19T04:07:07.760198+0000","last_fullsized":"2024-09-19T04:07:07.760198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:18.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294176+0000","last_change":"2024-09-19T04:04:46.358113+0000","last_active":"2024-09-19T04:06:43.294176+0000","last_peered":"2024-09-19T04:06:43.294176+0000","last_clean":"2024-09-19T04:06:43.294176+0000","last_became_active":"2024-09-19T04:04:46.356604+0000","last_became_peered":"2024-09-19T04:04:46.356604+0000","last_unstale":"2024-09-19T04:06:43.294176+0000","last_undegraded":"2024-09-19T04:06:43.294176+0000","last_fullsized":"2024-09-19T04:06:43.294176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:38.147738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912583+0000","last_change":"2024-09-19T04:04:46.354516+0000","last_active":"2024-09-19T04:06:42.912583+0000","last_peered":"2024-09-19T04:06:42.912583+0000","last_clean":"2024-09-19T04:06:42.912583+0000","last_became_active":"2024-09-19T04:04:46.354034+0000","last_became_peered":"2024-09-19T04:04:46.354034+0000","last_unstale":"2024-09-19T04:06:42.912583+0000","last_undegraded":"2024-09-19T04:06:42.912583+0000","last_fullsized":"2024-09-19T04:06:42.912583+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:21.946746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912926+0000","last_change":"2024-09-19T04:04:39.311324+0000","last_active":"2024-09-19T04:06:42.912926+0000","last_peered":"2024-09-19T04:06:42.912926+0000","last_clean":"2024-09-19T04:06:42.912926+0000","last_became_active":"2024-09-19T04:04:39.310977+0000","last_became_peered":"2024-09-19T04:04:39.310977+0000","last_unstale":"2024-09-19T04:06:42.912926+0000","last_undegraded":"2024-09-19T04:06:42.912926+0000","last_fullsized":"2024-09-19T04:06:42.912926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:43.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912352+0000","last_change":"2024-09-19T04:04:37.212773+0000","last_active":"2024-09-19T04:06:42.912352+0000","last_peered":"2024-09-19T04:06:42.912352+0000","last_clean":"2024-09-19T04:06:42.912352+0000","last_became_active":"2024-09-19T04:04:37.212405+0000","last_became_peered":"2024-09-19T04:04:37.212405+0000","last_unstale":"2024-09-19T04:06:42.912352+0000","last_undegraded":"2024-09-19T04:06:42.912352+0000","last_fullsized":"2024-09-19T04:06:42.912352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:22:29.020122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111297+0000","last_change":"2024-09-19T04:04:46.363052+0000","last_active":"2024-09-19T04:06:44.111297+0000","last_peered":"2024-09-19T04:06:44.111297+0000","last_clean":"2024-09-19T04:06:44.111297+0000","last_became_active":"2024-09-19T04:04:46.362588+0000","last_became_peered":"2024-09-19T04:04:46.362588+0000","last_unstale":"2024-09-19T04:06:44.111297+0000","last_undegraded":"2024-09-19T04:06:44.111297+0000","last_fullsized":"2024-09-19T04:06:44.111297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:52.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294114+0000","last_change":"2024-09-19T04:04:39.313686+0000","last_active":"2024-09-19T04:06:43.294114+0000","last_peered":"2024-09-19T04:06:43.294114+0000","last_clean":"2024-09-19T04:06:43.294114+0000","last_became_active":"2024-09-19T04:04:39.313386+0000","last_became_peered":"2024-09-19T04:04:39.313386+0000","last_unstale":"2024-09-19T04:06:43.294114+0000","last_undegraded":"2024-09-19T04:06:43.294114+0000","last_fullsized":"2024-09-19T04:06:43.294114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:08.274930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.908950+0000","last_change":"2024-09-19T04:04:37.207926+0000","last_active":"2024-09-19T04:06:42.908950+0000","last_peered":"2024-09-19T04:06:42.908950+0000","last_clean":"2024-09-19T04:06:42.908950+0000","last_became_active":"2024-09-19T04:04:37.207538+0000","last_became_peered":"2024-09-19T04:04:37.207538+0000","last_unstale":"2024-09-19T04:06:42.908950+0000","last_undegraded":"2024-09-19T04:06:42.908950+0000","last_fullsized":"2024-09-19T04:06:42.908950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:29.621638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914004+0000","last_change":"2024-09-19T04:04:37.212829+0000","last_active":"2024-09-19T04:06:42.914004+0000","last_peered":"2024-09-19T04:06:42.914004+0000","last_clean":"2024-09-19T04:06:42.914004+0000","last_became_active":"2024-09-19T04:04:37.212294+0000","last_became_peered":"2024-09-19T04:04:37.212294+0000","last_unstale":"2024-09-19T04:06:42.914004+0000","last_undegraded":"2024-09-19T04:06:42.914004+0000","last_fullsized":"2024-09-19T04:06:42.914004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:34.479708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912531+0000","last_change":"2024-09-19T04:04:39.328622+0000","last_active":"2024-09-19T04:06:42.912531+0000","last_peered":"2024-09-19T04:06:42.912531+0000","last_clean":"2024-09-19T04:06:42.912531+0000","last_became_active":"2024-09-19T04:04:39.328290+0000","last_became_peered":"2024-09-19T04:04:39.328290+0000","last_unstale":"2024-09-19T04:06:42.912531+0000","last_undegraded":"2024-09-19T04:06:42.912531+0000","last_fullsized":"2024-09-19T04:06:42.912531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:04.290767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914155+0000","last_change":"2024-09-19T04:04:46.362853+0000","last_active":"2024-09-19T04:06:42.914155+0000","last_peered":"2024-09-19T04:06:42.914155+0000","last_clean":"2024-09-19T04:06:42.914155+0000","last_became_active":"2024-09-19T04:04:46.362392+0000","last_became_peered":"2024-09-19T04:04:46.362392+0000","last_unstale":"2024-09-19T04:06:42.914155+0000","last_undegraded":"2024-09-19T04:06:42.914155+0000","last_fullsized":"2024-09-19T04:06:42.914155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:40.016502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:07:03.753495+0000","last_change":"2024-09-19T04:04:37.213248+0000","last_active":"2024-09-19T04:07:03.753495+0000","last_peered":"2024-09-19T04:07:03.753495+0000","last_clean":"2024-09-19T04:07:03.753495+0000","last_became_active":"2024-09-19T04:04:37.212639+0000","last_became_peered":"2024-09-19T04:04:37.212639+0000","last_unstale":"2024-09-19T04:07:03.753495+0000","last_undegraded":"2024-09-19T04:07:03.753495+0000","last_fullsized":"2024-09-19T04:07:03.753495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:36.180971+0000","last_clean_scrub_stamp":"2024-09-19T04:04:36.180971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:45:05.917792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916668+0000","last_change":"2024-09-19T04:04:39.330160+0000","last_active":"2024-09-19T04:06:42.916668+0000","last_peered":"2024-09-19T04:06:42.916668+0000","last_clean":"2024-09-19T04:06:42.916668+0000","last_became_active":"2024-09-19T04:04:39.329828+0000","last_became_peered":"2024-09-19T04:04:39.329828+0000","last_unstale":"2024-09-19T04:06:42.916668+0000","last_undegraded":"2024-09-19T04:06:42.916668+0000","last_fullsized":"2024-09-19T04:06:42.916668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:57.712550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914137+0000","last_change":"2024-09-19T04:04:46.360459+0000","last_active":"2024-09-19T04:06:42.914137+0000","last_peered":"2024-09-19T04:06:42.914137+0000","last_clean":"2024-09-19T04:06:42.914137+0000","last_became_active":"2024-09-19T04:04:46.359914+0000","last_became_peered":"2024-09-19T04:04:46.359914+0000","last_unstale":"2024-09-19T04:06:42.914137+0000","last_undegraded":"2024-09-19T04:06:42.914137+0000","last_fullsized":"2024-09-19T04:06:42.914137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:45.331773+0000","last_clean_scrub_stamp":"2024-09-19T04:04:45.331773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:30.380358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110927+0000","last_change":"2024-09-19T04:04:39.324847+0000","last_active":"2024-09-19T04:06:44.110927+0000","last_peered":"2024-09-19T04:06:44.110927+0000","last_clean":"2024-09-19T04:06:44.110927+0000","last_became_active":"2024-09-19T04:04:39.324402+0000","last_became_peered":"2024-09-19T04:04:39.324402+0000","last_unstale":"2024-09-19T04:06:44.110927+0000","last_undegraded":"2024-09-19T04:06:44.110927+0000","last_fullsized":"2024-09-19T04:06:44.110927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:41.996135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.918796+0000","last_change":"2024-09-19T04:06:42.918320+0000","last_active":"2024-09-19T04:06:42.918796+0000","last_peered":"2024-09-19T04:06:42.918796+0000","last_clean":"2024-09-19T04:06:42.918796+0000","last_became_active":"2024-09-19T04:06:42.917819+0000","last_became_peered":"2024-09-19T04:06:42.917819+0000","last_unstale":"2024-09-19T04:06:42.918796+0000","last_undegraded":"2024-09-19T04:06:42.918796+0000","last_fullsized":"2024-09-19T04:06:42.918796+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:02.885774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,8],"acting":[2,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914539+0000","last_change":"2024-09-19T04:04:39.318556+0000","last_active":"2024-09-19T04:06:42.914539+0000","last_peered":"2024-09-19T04:06:42.914539+0000","last_clean":"2024-09-19T04:06:42.914539+0000","last_became_active":"2024-09-19T04:04:39.318200+0000","last_became_peered":"2024-09-19T04:04:39.318200+0000","last_unstale":"2024-09-19T04:06:42.914539+0000","last_undegraded":"2024-09-19T04:06:42.914539+0000","last_fullsized":"2024-09-19T04:06:42.914539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:40.847440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912503+0000","last_change":"2024-09-19T04:04:39.320698+0000","last_active":"2024-09-19T04:06:42.912503+0000","last_peered":"2024-09-19T04:06:42.912503+0000","last_clean":"2024-09-19T04:06:42.912503+0000","last_became_active":"2024-09-19T04:04:39.320419+0000","last_became_peered":"2024-09-19T04:04:39.320419+0000","last_unstale":"2024-09-19T04:06:42.912503+0000","last_undegraded":"2024-09-19T04:06:42.912503+0000","last_fullsized":"2024-09-19T04:06:42.912503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:32.708394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909421+0000","last_change":"2024-09-19T04:04:39.319886+0000","last_active":"2024-09-19T04:06:42.909421+0000","last_peered":"2024-09-19T04:06:42.909421+0000","last_clean":"2024-09-19T04:06:42.909421+0000","last_became_active":"2024-09-19T04:04:39.319217+0000","last_became_peered":"2024-09-19T04:04:39.319217+0000","last_unstale":"2024-09-19T04:06:42.909421+0000","last_undegraded":"2024-09-19T04:06:42.909421+0000","last_fullsized":"2024-09-19T04:06:42.909421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:38.829608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111374+0000","last_change":"2024-09-19T04:04:39.327639+0000","last_active":"2024-09-19T04:06:44.111374+0000","last_peered":"2024-09-19T04:06:44.111374+0000","last_clean":"2024-09-19T04:06:44.111374+0000","last_became_active":"2024-09-19T04:04:39.327349+0000","last_became_peered":"2024-09-19T04:04:39.327349+0000","last_unstale":"2024-09-19T04:06:44.111374+0000","last_undegraded":"2024-09-19T04:06:44.111374+0000","last_fullsized":"2024-09-19T04:06:44.111374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:31.056609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914540+0000","last_change":"2024-09-19T04:04:39.320253+0000","last_active":"2024-09-19T04:06:42.914540+0000","last_peered":"2024-09-19T04:06:42.914540+0000","last_clean":"2024-09-19T04:06:42.914540+0000","last_became_active":"2024-09-19T04:04:39.319851+0000","last_became_peered":"2024-09-19T04:04:39.319851+0000","last_unstale":"2024-09-19T04:06:42.914540+0000","last_undegraded":"2024-09-19T04:06:42.914540+0000","last_fullsized":"2024-09-19T04:06:42.914540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:53.631216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294843+0000","last_change":"2024-09-19T04:04:39.322779+0000","last_active":"2024-09-19T04:06:43.294843+0000","last_peered":"2024-09-19T04:06:43.294843+0000","last_clean":"2024-09-19T04:06:43.294843+0000","last_became_active":"2024-09-19T04:04:39.322498+0000","last_became_peered":"2024-09-19T04:04:39.322498+0000","last_unstale":"2024-09-19T04:06:43.294843+0000","last_undegraded":"2024-09-19T04:06:43.294843+0000","last_fullsized":"2024-09-19T04:06:43.294843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:24:11.956002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.911989+0000","last_change":"2024-09-19T04:04:39.322759+0000","last_active":"2024-09-19T04:06:42.911989+0000","last_peered":"2024-09-19T04:06:42.911989+0000","last_clean":"2024-09-19T04:06:42.911989+0000","last_became_active":"2024-09-19T04:04:39.322429+0000","last_became_peered":"2024-09-19T04:04:39.322429+0000","last_unstale":"2024-09-19T04:06:42.911989+0000","last_undegraded":"2024-09-19T04:06:42.911989+0000","last_fullsized":"2024-09-19T04:06:42.911989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:36.069931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294608+0000","last_change":"2024-09-19T04:04:39.320078+0000","last_active":"2024-09-19T04:06:43.294608+0000","last_peered":"2024-09-19T04:06:43.294608+0000","last_clean":"2024-09-19T04:06:43.294608+0000","last_became_active":"2024-09-19T04:04:39.319755+0000","last_became_peered":"2024-09-19T04:04:39.319755+0000","last_unstale":"2024-09-19T04:06:43.294608+0000","last_undegraded":"2024-09-19T04:06:43.294608+0000","last_fullsized":"2024-09-19T04:06:43.294608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:07.651712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111342+0000","last_change":"2024-09-19T04:04:39.319839+0000","last_active":"2024-09-19T04:06:44.111342+0000","last_peered":"2024-09-19T04:06:44.111342+0000","last_clean":"2024-09-19T04:06:44.111342+0000","last_became_active":"2024-09-19T04:04:39.319527+0000","last_became_peered":"2024-09-19T04:04:39.319527+0000","last_unstale":"2024-09-19T04:06:44.111342+0000","last_undegraded":"2024-09-19T04:06:44.111342+0000","last_fullsized":"2024-09-19T04:06:44.111342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:31.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909471+0000","last_change":"2024-09-19T04:04:39.315975+0000","last_active":"2024-09-19T04:06:42.909471+0000","last_peered":"2024-09-19T04:06:42.909471+0000","last_clean":"2024-09-19T04:06:42.909471+0000","last_became_active":"2024-09-19T04:04:39.315671+0000","last_became_peered":"2024-09-19T04:04:39.315671+0000","last_unstale":"2024-09-19T04:06:42.909471+0000","last_undegraded":"2024-09-19T04:06:42.909471+0000","last_fullsized":"2024-09-19T04:06:42.909471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:16.349853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111082+0000","last_change":"2024-09-19T04:04:39.316278+0000","last_active":"2024-09-19T04:06:44.111082+0000","last_peered":"2024-09-19T04:06:44.111082+0000","last_clean":"2024-09-19T04:06:44.111082+0000","last_became_active":"2024-09-19T04:04:39.315876+0000","last_became_peered":"2024-09-19T04:04:39.315876+0000","last_unstale":"2024-09-19T04:06:44.111082+0000","last_undegraded":"2024-09-19T04:06:44.111082+0000","last_fullsized":"2024-09-19T04:06:44.111082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:23.544405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914652+0000","last_change":"2024-09-19T04:04:39.318952+0000","last_active":"2024-09-19T04:06:42.914652+0000","last_peered":"2024-09-19T04:06:42.914652+0000","last_clean":"2024-09-19T04:06:42.914652+0000","last_became_active":"2024-09-19T04:04:39.318598+0000","last_became_peered":"2024-09-19T04:04:39.318598+0000","last_unstale":"2024-09-19T04:06:42.914652+0000","last_undegraded":"2024-09-19T04:06:42.914652+0000","last_fullsized":"2024-09-19T04:06:42.914652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:12.654611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913733+0000","last_change":"2024-09-19T04:04:39.311334+0000","last_active":"2024-09-19T04:06:42.913733+0000","last_peered":"2024-09-19T04:06:42.913733+0000","last_clean":"2024-09-19T04:06:42.913733+0000","last_became_active":"2024-09-19T04:04:39.311038+0000","last_became_peered":"2024-09-19T04:04:39.311038+0000","last_unstale":"2024-09-19T04:06:42.913733+0000","last_undegraded":"2024-09-19T04:06:42.913733+0000","last_fullsized":"2024-09-19T04:06:42.913733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:41.655363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111491+0000","last_change":"2024-09-19T04:04:39.319529+0000","last_active":"2024-09-19T04:06:44.111491+0000","last_peered":"2024-09-19T04:06:44.111491+0000","last_clean":"2024-09-19T04:06:44.111491+0000","last_became_active":"2024-09-19T04:04:39.318787+0000","last_became_peered":"2024-09-19T04:04:39.318787+0000","last_unstale":"2024-09-19T04:06:44.111491+0000","last_undegraded":"2024-09-19T04:06:44.111491+0000","last_fullsized":"2024-09-19T04:06:44.111491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.209422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.110641+0000","last_change":"2024-09-19T04:04:39.320320+0000","last_active":"2024-09-19T04:06:44.110641+0000","last_peered":"2024-09-19T04:06:44.110641+0000","last_clean":"2024-09-19T04:06:44.110641+0000","last_became_active":"2024-09-19T04:04:39.319091+0000","last_became_peered":"2024-09-19T04:04:39.319091+0000","last_unstale":"2024-09-19T04:06:44.110641+0000","last_undegraded":"2024-09-19T04:06:44.110641+0000","last_fullsized":"2024-09-19T04:06:44.110641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:20:39.495079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912247+0000","last_change":"2024-09-19T04:04:39.323270+0000","last_active":"2024-09-19T04:06:42.912247+0000","last_peered":"2024-09-19T04:06:42.912247+0000","last_clean":"2024-09-19T04:06:42.912247+0000","last_became_active":"2024-09-19T04:04:39.322890+0000","last_became_peered":"2024-09-19T04:04:39.322890+0000","last_unstale":"2024-09-19T04:06:42.912247+0000","last_undegraded":"2024-09-19T04:06:42.912247+0000","last_fullsized":"2024-09-19T04:06:42.912247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:07.311654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912217+0000","last_change":"2024-09-19T04:04:39.317098+0000","last_active":"2024-09-19T04:06:42.912217+0000","last_peered":"2024-09-19T04:06:42.912217+0000","last_clean":"2024-09-19T04:06:42.912217+0000","last_became_active":"2024-09-19T04:04:39.316792+0000","last_became_peered":"2024-09-19T04:04:39.316792+0000","last_unstale":"2024-09-19T04:06:42.912217+0000","last_undegraded":"2024-09-19T04:06:42.912217+0000","last_fullsized":"2024-09-19T04:06:42.912217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.571149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.909396+0000","last_change":"2024-09-19T04:04:39.326473+0000","last_active":"2024-09-19T04:06:42.909396+0000","last_peered":"2024-09-19T04:06:42.909396+0000","last_clean":"2024-09-19T04:06:42.909396+0000","last_became_active":"2024-09-19T04:04:39.325506+0000","last_became_peered":"2024-09-19T04:04:39.325506+0000","last_unstale":"2024-09-19T04:06:42.909396+0000","last_undegraded":"2024-09-19T04:06:42.909396+0000","last_fullsized":"2024-09-19T04:06:42.909396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:40.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915242+0000","last_change":"2024-09-19T04:04:39.323536+0000","last_active":"2024-09-19T04:06:42.915242+0000","last_peered":"2024-09-19T04:06:42.915242+0000","last_clean":"2024-09-19T04:06:42.915242+0000","last_became_active":"2024-09-19T04:04:39.323224+0000","last_became_peered":"2024-09-19T04:04:39.323224+0000","last_unstale":"2024-09-19T04:06:42.915242+0000","last_undegraded":"2024-09-19T04:06:42.915242+0000","last_fullsized":"2024-09-19T04:06:42.915242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:54.640599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.419156+0000","last_change":"2024-09-19T04:04:39.323802+0000","last_active":"2024-09-19T04:06:43.419156+0000","last_peered":"2024-09-19T04:06:43.419156+0000","last_clean":"2024-09-19T04:06:43.419156+0000","last_became_active":"2024-09-19T04:04:39.323031+0000","last_became_peered":"2024-09-19T04:04:39.323031+0000","last_unstale":"2024-09-19T04:06:43.419156+0000","last_undegraded":"2024-09-19T04:06:43.419156+0000","last_fullsized":"2024-09-19T04:06:43.419156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:12.467182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.914673+0000","last_change":"2024-09-19T04:04:39.324870+0000","last_active":"2024-09-19T04:06:42.914673+0000","last_peered":"2024-09-19T04:06:42.914673+0000","last_clean":"2024-09-19T04:06:42.914673+0000","last_became_active":"2024-09-19T04:04:39.324551+0000","last_became_peered":"2024-09-19T04:04:39.324551+0000","last_unstale":"2024-09-19T04:06:42.914673+0000","last_undegraded":"2024-09-19T04:06:42.914673+0000","last_fullsized":"2024-09-19T04:06:42.914673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:34.766437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111230+0000","last_change":"2024-09-19T04:04:39.329086+0000","last_active":"2024-09-19T04:06:44.111230+0000","last_peered":"2024-09-19T04:06:44.111230+0000","last_clean":"2024-09-19T04:06:44.111230+0000","last_became_active":"2024-09-19T04:04:39.328651+0000","last_became_peered":"2024-09-19T04:04:39.328651+0000","last_unstale":"2024-09-19T04:06:44.111230+0000","last_undegraded":"2024-09-19T04:06:44.111230+0000","last_fullsized":"2024-09-19T04:06:44.111230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:12.079162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.294120+0000","last_change":"2024-09-19T04:04:39.325427+0000","last_active":"2024-09-19T04:06:43.294120+0000","last_peered":"2024-09-19T04:06:43.294120+0000","last_clean":"2024-09-19T04:06:43.294120+0000","last_became_active":"2024-09-19T04:04:39.325113+0000","last_became_peered":"2024-09-19T04:04:39.325113+0000","last_unstale":"2024-09-19T04:06:43.294120+0000","last_undegraded":"2024-09-19T04:06:43.294120+0000","last_fullsized":"2024-09-19T04:06:43.294120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:15.844503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:43.418560+0000","last_change":"2024-09-19T04:04:39.325032+0000","last_active":"2024-09-19T04:06:43.418560+0000","last_peered":"2024-09-19T04:06:43.418560+0000","last_clean":"2024-09-19T04:06:43.418560+0000","last_became_active":"2024-09-19T04:04:39.324618+0000","last_became_peered":"2024-09-19T04:04:39.324618+0000","last_unstale":"2024-09-19T04:06:43.418560+0000","last_undegraded":"2024-09-19T04:06:43.418560+0000","last_fullsized":"2024-09-19T04:06:43.418560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:58.748613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.913935+0000","last_change":"2024-09-19T04:04:39.318336+0000","last_active":"2024-09-19T04:06:42.913935+0000","last_peered":"2024-09-19T04:06:42.913935+0000","last_clean":"2024-09-19T04:06:42.913935+0000","last_became_active":"2024-09-19T04:04:39.317032+0000","last_became_peered":"2024-09-19T04:04:39.317032+0000","last_unstale":"2024-09-19T04:06:42.913935+0000","last_undegraded":"2024-09-19T04:06:42.913935+0000","last_fullsized":"2024-09-19T04:06:42.913935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:04.730265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.916560+0000","last_change":"2024-09-19T04:04:39.321909+0000","last_active":"2024-09-19T04:06:42.916560+0000","last_peered":"2024-09-19T04:06:42.916560+0000","last_clean":"2024-09-19T04:06:42.916560+0000","last_became_active":"2024-09-19T04:04:39.320522+0000","last_became_peered":"2024-09-19T04:04:39.320522+0000","last_unstale":"2024-09-19T04:06:42.916560+0000","last_undegraded":"2024-09-19T04:06:42.916560+0000","last_fullsized":"2024-09-19T04:06:42.916560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:57.270998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.915032+0000","last_change":"2024-09-19T04:04:39.324610+0000","last_active":"2024-09-19T04:06:42.915032+0000","last_peered":"2024-09-19T04:06:42.915032+0000","last_clean":"2024-09-19T04:06:42.915032+0000","last_became_active":"2024-09-19T04:04:39.324190+0000","last_became_peered":"2024-09-19T04:04:39.324190+0000","last_unstale":"2024-09-19T04:06:42.915032+0000","last_undegraded":"2024-09-19T04:06:42.915032+0000","last_fullsized":"2024-09-19T04:06:42.915032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:51.418210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912621+0000","last_change":"2024-09-19T04:04:39.324220+0000","last_active":"2024-09-19T04:06:42.912621+0000","last_peered":"2024-09-19T04:06:42.912621+0000","last_clean":"2024-09-19T04:06:42.912621+0000","last_became_active":"2024-09-19T04:04:39.323890+0000","last_became_peered":"2024-09-19T04:04:39.323890+0000","last_unstale":"2024-09-19T04:06:42.912621+0000","last_undegraded":"2024-09-19T04:06:42.912621+0000","last_fullsized":"2024-09-19T04:06:42.912621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:15.475088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:42.912692+0000","last_change":"2024-09-19T04:04:39.317240+0000","last_active":"2024-09-19T04:06:42.912692+0000","last_peered":"2024-09-19T04:06:42.912692+0000","last_clean":"2024-09-19T04:06:42.912692+0000","last_became_active":"2024-09-19T04:04:39.316740+0000","last_became_peered":"2024-09-19T04:04:39.316740+0000","last_unstale":"2024-09-19T04:06:42.912692+0000","last_undegraded":"2024-09-19T04:06:42.912692+0000","last_fullsized":"2024-09-19T04:06:42.912692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:53:46.705467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T04:06:44.111449+0000","last_change":"2024-09-19T04:04:39.319665+0000","last_active":"2024-09-19T04:06:44.111449+0000","last_peered":"2024-09-19T04:06:44.111449+0000","last_clean":"2024-09-19T04:06:44.111449+0000","last_became_active":"2024-09-19T04:04:39.319227+0000","last_became_peered":"2024-09-19T04:04:39.319227+0000","last_unstale":"2024-09-19T04:06:44.111449+0000","last_undegraded":"2024-09-19T04:06:44.111449+0000","last_fullsized":"2024-09-19T04:06:44.111449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:04:38.289376+0000","last_clean_scrub_stamp":"2024-09-19T04:04:38.289376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:45.581254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78183,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1089,"num_read_kb":635,"num_write":263,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":60960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":226263,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248101,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481835,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1036288,"data_stored":529087,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710768,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1191936,"data_stored":1186800,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874294,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":533091,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135931,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":529351,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1105920,"data_stored":1126321,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528389,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":525033,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1121369,"data_compressed":65321,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":599492,"data_compressed":39195,"data_compressed_allocated":53248,"data_compressed_original":139264,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":524207,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":526057,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1101824,"data_stored":588388,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":525955,"data_compressed":29410,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34273,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":71619,"data_compressed":9785,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:07:09.375 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:07:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:10.026 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:09 smithi019 ceph-mon[30938]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T04:07:10.026 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:09 smithi019 ceph-mon[30938]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:10.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 -- 172.21.15.19:0/3035778277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0102790 msgr2=0x7fc0e01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:10.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/3035778277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0102790 0x7fc0e01067e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009900 tx=0x7fc0c802a410 comp rx=0 tx=0).stop 2024-09-19T04:07:10.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 -- 172.21.15.19:0/3035778277 shutdown_connections 2024-09-19T04:07:10.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/3035778277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0e0106f10 0x7fc0e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/3035778277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0102790 0x7fc0e01067e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/3035778277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0e0101de0 0x7fc0e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 -- 172.21.15.19:0/3035778277 >> 172.21.15.19:0/3035778277 conn(0x7fc0e00fd8c0 msgr2=0x7fc0e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 -- 172.21.15.19:0/3035778277 shutdown_connections 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.109+0000 7fc0e56b8640 1 -- 172.21.15.19:0/3035778277 wait complete. 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0e56b8640 1 Processor -- start 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0e56b8640 1 -- start start 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0e56b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0101de0 0x7fc0e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0e56b8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0e0102790 0x7fc0e0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0e56b8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0e0106f10 0x7fc0e0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:10.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0e56b8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0111f70 con 0x7fc0e0101de0 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0e56b8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0111df0 con 0x7fc0e0102790 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0e56b8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01120f0 con 0x7fc0e0106f10 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0101de0 0x7fc0e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0101de0 0x7fc0e00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38978/0 (socket says 172.21.15.19:38978) 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0deffd640 1 -- 172.21.15.19:0/1656113615 learned_addr learned my addr 172.21.15.19:0/1656113615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.110+0000 7fc0df7fe640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0e0106f10 0x7fc0e0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0de7fc640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0e0102790 0x7fc0e0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0deffd640 1 -- 172.21.15.19:0/1656113615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0e0106f10 msgr2=0x7fc0e0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0deffd640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0e0106f10 0x7fc0e0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0deffd640 1 -- 172.21.15.19:0/1656113615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0e0102790 msgr2=0x7fc0e0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0deffd640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0e0102790 0x7fc0e0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0deffd640 1 -- 172.21.15.19:0/1656113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e01a6900 con 0x7fc0e0101de0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0de7fc640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0e0102790 0x7fc0e0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0deffd640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0101de0 0x7fc0e00744f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000d950 tx=0x7fc0d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0d7fff640 1 -- 172.21.15.19:0/1656113615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0017030 con 0x7fc0e0101de0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e01a6bf0 con 0x7fc0e0101de0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0d7fff640 1 -- 172.21.15.19:0/1656113615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d0004020 con 0x7fc0e0101de0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0d7fff640 1 -- 172.21.15.19:0/1656113615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00047a0 con 0x7fc0e0101de0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.111+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e01a7180 con 0x7fc0e0101de0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.112+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e01021c0 con 0x7fc0e0101de0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.113+0000 7fc0d7fff640 1 -- 172.21.15.19:0/1656113615 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fc0d00041d0 con 0x7fc0e0101de0 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.114+0000 7fc0d7fff640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc0cc0a32d0 0x7fc0cc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.114+0000 7fc0d7fff640 1 -- 172.21.15.19:0/1656113615 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fc0d00cf820 con 0x7fc0e0101de0 2024-09-19T04:07:10.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.114+0000 7fc0de7fc640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc0cc0a32d0 0x7fc0cc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:10.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.115+0000 7fc0de7fc640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc0cc0a32d0 0x7fc0cc0a5790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc0e01087d0 tx=0x7fc0c8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:10.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.118+0000 7fc0d7fff640 1 -- 172.21.15.19:0/1656113615 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc0d009c040 con 0x7fc0e0101de0 2024-09-19T04:07:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.335+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0e010a150 con 0x7fc0e0101de0 2024-09-19T04:07:10.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.336+0000 7fc0d7fff640 1 -- 172.21.15.19:0/1656113615 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7fc0d009b940 con 0x7fc0e0101de0 2024-09-19T04:07:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:10.338 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:07:10.338 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:07:10.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc0cc0a32d0 msgr2=0x7fc0cc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc0cc0a32d0 0x7fc0cc0a5790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc0e01087d0 tx=0x7fc0c8005a20 comp rx=0 tx=0).stop 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0101de0 msgr2=0x7fc0e00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0101de0 0x7fc0e00744f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000d950 tx=0x7fc0d000de10 comp rx=0 tx=0).stop 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 shutdown_connections 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc0cc0a32d0 0x7fc0cc0a5790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0e0106f10 0x7fc0e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc0e0102790 0x7fc0e0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 --2- 172.21.15.19:0/1656113615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0e0101de0 0x7fc0e00744f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 >> 172.21.15.19:0/1656113615 conn(0x7fc0e00fd8c0 msgr2=0x7fc0e00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 shutdown_connections 2024-09-19T04:07:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:10.338+0000 7fc0e56b8640 1 -- 172.21.15.19:0/1656113615 wait complete. 2024-09-19T04:07:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:09 smithi111 ceph-mon[35365]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T04:07:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:09 smithi111 ceph-mon[35365]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:09 smithi196 ceph-mon[35731]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T04:07:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:09 smithi196 ceph-mon[35731]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:07:11.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:07:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:11.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:10 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1656113615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:11.409 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:10 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1656113615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:11.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:10 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1656113615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 -- 172.21.15.19:0/338530253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c106f10 msgr2=0x7f1d8c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/338530253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c106f10 0x7f1d8c10ea90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009ca0 tx=0x7f1d8402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 -- 172.21.15.19:0/338530253 shutdown_connections 2024-09-19T04:07:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/338530253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c106f10 0x7f1d8c10ea90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/338530253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d8c102790 0x7f1d8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/338530253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d8c101de0 0x7f1d8c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 -- 172.21.15.19:0/338530253 >> 172.21.15.19:0/338530253 conn(0x7f1d8c0fd8c0 msgr2=0x7f1d8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 -- 172.21.15.19:0/338530253 shutdown_connections 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.746+0000 7f1d92ab9640 1 -- 172.21.15.19:0/338530253 wait complete. 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.747+0000 7f1d92ab9640 1 Processor -- start 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.747+0000 7f1d92ab9640 1 -- start start 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.747+0000 7f1d92ab9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c101de0 0x7f1d8c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.747+0000 7f1d92ab9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d8c102790 0x7f1d8c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.747+0000 7f1d92ab9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d8c106f10 0x7f1d8c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.747+0000 7f1d92ab9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c111fb0 con 0x7f1d8c101de0 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.747+0000 7f1d92ab9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c111e30 con 0x7f1d8c106f10 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.747+0000 7f1d92ab9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c112130 con 0x7f1d8c102790 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c101de0 0x7f1d8c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c101de0 0x7f1d8c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57190/0 (socket says 172.21.15.19:57190) 2024-09-19T04:07:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 -- 172.21.15.19:0/2341580127 learned_addr learned my addr 172.21.15.19:0/2341580127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d91056640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d8c106f10 0x7f1d8c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d8bfff640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d8c102790 0x7f1d8c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 -- 172.21.15.19:0/2341580127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d8c102790 msgr2=0x7f1d8c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d8c102790 0x7f1d8c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 -- 172.21.15.19:0/2341580127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d8c106f10 msgr2=0x7f1d8c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d8c106f10 0x7f1d8c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 -- 172.21.15.19:0/2341580127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c19d440 con 0x7f1d8c101de0 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d90855640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c101de0 0x7f1d8c198490 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000b4a0 tx=0x7f1d8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d89ffb640 1 -- 172.21.15.19:0/2341580127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80014070 con 0x7f1d8c101de0 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8c19d730 con 0x7f1d8c101de0 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d89ffb640 1 -- 172.21.15.19:0/2341580127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d80010af0 con 0x7f1d8c101de0 2024-09-19T04:07:11.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d89ffb640 1 -- 172.21.15.19:0/2341580127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80017d10 con 0x7f1d8c101de0 2024-09-19T04:07:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.748+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8c1a5010 con 0x7f1d8c101de0 2024-09-19T04:07:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.750+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8c1021c0 con 0x7f1d8c101de0 2024-09-19T04:07:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.751+0000 7f1d89ffb640 1 -- 172.21.15.19:0/2341580127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f1d80017070 con 0x7f1d8c101de0 2024-09-19T04:07:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.751+0000 7f1d89ffb640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d7c0a32d0 0x7f1d7c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.752+0000 7f1d8bfff640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d7c0a32d0 0x7f1d7c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.752+0000 7f1d89ffb640 1 -- 172.21.15.19:0/2341580127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f1d800cf210 con 0x7f1d8c101de0 2024-09-19T04:07:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.752+0000 7f1d8bfff640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d7c0a32d0 0x7f1d7c0a5790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1d78005fd0 tx=0x7f1d78005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.756+0000 7f1d89ffb640 1 -- 172.21.15.19:0/2341580127 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d8009bae0 con 0x7f1d8c101de0 2024-09-19T04:07:11.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.969+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d8c10a150 con 0x7f1d8c101de0 2024-09-19T04:07:11.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.969+0000 7f1d89ffb640 1 -- 172.21.15.19:0/2341580127 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7f1d8009b3e0 con 0x7f1d8c101de0 2024-09-19T04:07:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:11.971 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:07:11.971 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:07:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d7c0a32d0 msgr2=0x7f1d7c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d7c0a32d0 0x7f1d7c0a5790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1d78005fd0 tx=0x7f1d78005e20 comp rx=0 tx=0).stop 2024-09-19T04:07:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c101de0 msgr2=0x7f1d8c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c101de0 0x7f1d8c198490 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000b4a0 tx=0x7f1d8000b960 comp rx=0 tx=0).stop 2024-09-19T04:07:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 shutdown_connections 2024-09-19T04:07:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f1d7c0a32d0 0x7f1d7c0a5790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d8c106f10 0x7f1d8c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d8c102790 0x7f1d8c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 --2- 172.21.15.19:0/2341580127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d8c101de0 0x7f1d8c198490 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 >> 172.21.15.19:0/2341580127 conn(0x7f1d8c0fd8c0 msgr2=0x7f1d8c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 shutdown_connections 2024-09-19T04:07:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:11.972+0000 7f1d92ab9640 1 -- 172.21.15.19:0/2341580127 wait complete. 2024-09-19T04:07:12.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:11 smithi111 ceph-mon[35365]: pgmap v368: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T04:07:12.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:11 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2341580127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:12.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:11 smithi019 ceph-mon[30938]: pgmap v368: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T04:07:12.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:11 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2341580127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:12.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:11 smithi196 ceph-mon[35731]: pgmap v368: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T04:07:12.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:11 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2341580127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:12.635 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:07:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 -- 172.21.15.19:0/1807599738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6010c2f0 msgr2=0x7f0e6010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 --2- 172.21.15.19:0/1807599738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6010c2f0 0x7f0e6010e790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009ca0 tx=0x7f0e5802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 -- 172.21.15.19:0/1807599738 shutdown_connections 2024-09-19T04:07:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 --2- 172.21.15.19:0/1807599738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6010c2f0 0x7f0e6010e790 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 --2- 172.21.15.19:0/1807599738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e601037a0 0x7f0e6010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 --2- 172.21.15.19:0/1807599738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e60102df0 0x7f0e601031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 -- 172.21.15.19:0/1807599738 >> 172.21.15.19:0/1807599738 conn(0x7f0e600fc7d0 msgr2=0x7f0e600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 -- 172.21.15.19:0/1807599738 shutdown_connections 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.385+0000 7f0e668af640 1 -- 172.21.15.19:0/1807599738 wait complete. 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e668af640 1 Processor -- start 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e668af640 1 -- start start 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e668af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102df0 0x7f0e60198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e668af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e601037a0 0x7f0e60198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e668af640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e6010c2f0 0x7f0e6019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e668af640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60112150 con 0x7f0e601037a0 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e668af640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60111fd0 con 0x7f0e6010c2f0 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e668af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601122d0 con 0x7f0e60102df0 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e601037a0 0x7f0e60198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e601037a0 0x7f0e60198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57212/0 (socket says 172.21.15.19:57212) 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.386+0000 7f0e5f7fe640 1 -- 172.21.15.19:0/1975549675 learned_addr learned my addr 172.21.15.19:0/1975549675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e64e4c640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e6010c2f0 0x7f0e6019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5ffff640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102df0 0x7f0e60198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5f7fe640 1 -- 172.21.15.19:0/1975549675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102df0 msgr2=0x7f0e60198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5f7fe640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102df0 0x7f0e60198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5f7fe640 1 -- 172.21.15.19:0/1975549675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e6010c2f0 msgr2=0x7f0e6019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5f7fe640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e6010c2f0 0x7f0e6019cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5f7fe640 1 -- 172.21.15.19:0/1975549675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6019d5c0 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5ffff640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102df0 0x7f0e60198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5f7fe640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e601037a0 0x7f0e60198b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006fd0 tx=0x7f0e5400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5d7fa640 1 -- 172.21.15.19:0/1975549675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54011830 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5d7fa640 1 -- 172.21.15.19:0/1975549675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e54016070 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6019d8b0 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e5d7fa640 1 -- 172.21.15.19:0/1975549675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400f2e0 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.387+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e601a5190 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.388+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e601031d0 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.389+0000 7f0e5d7fa640 1 -- 172.21.15.19:0/1975549675 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f0e5401b460 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.390+0000 7f0e5d7fa640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e480a32d0 0x7f0e480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.390+0000 7f0e5d7fa640 1 -- 172.21.15.19:0/1975549675 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f0e540cf3d0 con 0x7f0e601037a0 2024-09-19T04:07:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.390+0000 7f0e5ffff640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e480a32d0 0x7f0e480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.390+0000 7f0e5ffff640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e480a32d0 0x7f0e480a5790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0e500058c0 tx=0x7f0e50005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.396+0000 7f0e5d7fa640 1 -- 172.21.15.19:0/1975549675 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e5409bd00 con 0x7f0e601037a0 2024-09-19T04:07:13.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.573+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e601072c0 con 0x7f0e601037a0 2024-09-19T04:07:13.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.575+0000 7f0e5d7fa640 1 -- 172.21.15.19:0/1975549675 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24842 (secure 0 0 0) 0x7f0e5409b600 con 0x7f0e601037a0 2024-09-19T04:07:13.577 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:13.577 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":93,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:06:42.902449+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:07:13.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e480a32d0 msgr2=0x7f0e480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e480a32d0 0x7f0e480a5790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0e500058c0 tx=0x7f0e50005d30 comp rx=0 tx=0).stop 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e601037a0 msgr2=0x7f0e60198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e601037a0 0x7f0e60198b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006fd0 tx=0x7f0e5400b8e0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 shutdown_connections 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f0e480a32d0 0x7f0e480a5790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e6010c2f0 0x7f0e6019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e601037a0 0x7f0e60198b50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 --2- 172.21.15.19:0/1975549675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e60102df0 0x7f0e60198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 >> 172.21.15.19:0/1975549675 conn(0x7f0e600fc7d0 msgr2=0x7f0e600fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 shutdown_connections 2024-09-19T04:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:13.578+0000 7f0e668af640 1 -- 172.21.15.19:0/1975549675 wait complete. 2024-09-19T04:07:14.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:13 smithi019 ceph-mon[30938]: pgmap v369: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T04:07:14.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:13 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1975549675' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:07:14.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T04:07:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:13 smithi111 ceph-mon[35365]: pgmap v369: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T04:07:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:13 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1975549675' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:07:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:13 smithi196 ceph-mon[35731]: pgmap v369: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T04:07:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:13 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1975549675' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:07:14.662 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:15.966 INFO:teuthology.orchestra.run.smithi019.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T04:07:15.966 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T04:07:15.967 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T04:07:15.967 DEBUG:tasks.quiescer:Yielding 2024-09-19T04:07:15.967 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T04:07:15.983 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T04:07:15.983 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T04:07:15.983 INFO:tasks.workunit:timeout=3h 2024-09-19T04:07:15.983 INFO:tasks.workunit:cleanup=True 2024-09-19T04:07:15.984 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:07:15.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-19 04:05:30.223059523 +0000 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-19 04:05:30.223059523 +0000 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-19 04:07:02.802742096 +0000 2024-09-19T04:07:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-19 04:05:30.223059523 +0000 2024-09-19T04:07:16.004 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T04:07:16.004 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T04:07:16.079 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T04:07:16.135 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T04:07:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:16.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:15 smithi019 ceph-mon[30938]: pgmap v370: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T04:07:16.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:15 smithi111 ceph-mon[35365]: pgmap v370: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T04:07:16.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:15 smithi196 ceph-mon[35731]: pgmap v370: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 -- 172.21.15.19:0/536219285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 msgr2=0x7fed2c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 --2- 172.21.15.19:0/536219285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 0x7fed2c10da40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed24009ca0 tx=0x7fed2402a950 comp rx=0 tx=0).stop 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 -- 172.21.15.19:0/536219285 shutdown_connections 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 --2- 172.21.15.19:0/536219285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 0x7fed2c10da40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 --2- 172.21.15.19:0/536219285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed2c102570 0x7fed2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 --2- 172.21.15.19:0/536219285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c100e20 0x7fed2c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 -- 172.21.15.19:0/536219285 >> 172.21.15.19:0/536219285 conn(0x7fed2c0fc7d0 msgr2=0x7fed2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 -- 172.21.15.19:0/536219285 shutdown_connections 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.873+0000 7fed315f2640 1 -- 172.21.15.19:0/536219285 wait complete. 2024-09-19T04:07:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed315f2640 1 Processor -- start 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed315f2640 1 -- start start 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed315f2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c100e20 0x7fed2c19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed315f2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed2c102570 0x7fed2c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed315f2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 0x7fed2c1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed315f2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c110f80 con 0x7fed2c102ef0 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed315f2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c110e00 con 0x7fed2c102570 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed315f2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c111100 con 0x7fed2c100e20 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 0x7fed2c1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 0x7fed2c1963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57226/0 (socket says 172.21.15.19:57226) 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed2b7fe640 1 -- 172.21.15.19:0/1737673868 learned_addr learned my addr 172.21.15.19:0/1737673868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed227fc640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed2c102570 0x7fed2c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.874+0000 7fed2affd640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c100e20 0x7fed2c19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed2b7fe640 1 -- 172.21.15.19:0/1737673868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c100e20 msgr2=0x7fed2c19e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed2b7fe640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c100e20 0x7fed2c19e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed2b7fe640 1 -- 172.21.15.19:0/1737673868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed2c102570 msgr2=0x7fed2c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed2b7fe640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed2c102570 0x7fed2c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed2b7fe640 1 -- 172.21.15.19:0/1737673868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c196cb0 con 0x7fed2c102ef0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed2affd640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c100e20 0x7fed2c19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed2b7fe640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 0x7fed2c1963f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fed24006010 tx=0x7fed2402f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed28ff9640 1 -- 172.21.15.19:0/1737673868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24034040 con 0x7fed2c102ef0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed28ff9640 1 -- 172.21.15.19:0/1737673868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24003e00 con 0x7fed2c102ef0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c196f40 con 0x7fed2c102ef0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed28ff9640 1 -- 172.21.15.19:0/1737673868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24045480 con 0x7fed2c102ef0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.875+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1b3e00 con 0x7fed2c102ef0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.876+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c1021b0 con 0x7fed2c102ef0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.877+0000 7fed28ff9640 1 -- 172.21.15.19:0/1737673868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fed24037020 con 0x7fed2c102ef0 2024-09-19T04:07:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.878+0000 7fed28ff9640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fed180a32d0 0x7fed180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.878+0000 7fed2affd640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fed180a32d0 0x7fed180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.878+0000 7fed28ff9640 1 -- 172.21.15.19:0/1737673868 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fed240f0e60 con 0x7fed2c102ef0 2024-09-19T04:07:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.879+0000 7fed2affd640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fed180a32d0 0x7fed180a5790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0058c0 tx=0x7fed1c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:16.882+0000 7fed28ff9640 1 -- 172.21.15.19:0/1737673868 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed240bd790 con 0x7fed2c102ef0 2024-09-19T04:07:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.096+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed2c101200 con 0x7fed2c102ef0 2024-09-19T04:07:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.097+0000 7fed28ff9640 1 -- 172.21.15.19:0/1737673868 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7fed240bd090 con 0x7fed2c102ef0 2024-09-19T04:07:17.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:17.098 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:07:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:07:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fed180a32d0 msgr2=0x7fed180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fed180a32d0 0x7fed180a5790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0058c0 tx=0x7fed1c005ec0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 msgr2=0x7fed2c1963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 0x7fed2c1963f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fed24006010 tx=0x7fed2402f6f0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 shutdown_connections 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fed180a32d0 0x7fed180a5790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed2c102ef0 0x7fed2c1963f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed2c102570 0x7fed2c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 --2- 172.21.15.19:0/1737673868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c100e20 0x7fed2c19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 >> 172.21.15.19:0/1737673868 conn(0x7fed2c0fc7d0 msgr2=0x7fed2c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 shutdown_connections 2024-09-19T04:07:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:17.099+0000 7fed315f2640 1 -- 172.21.15.19:0/1737673868 wait complete. 2024-09-19T04:07:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 -- 172.21.15.19:0/1230749180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c100cd0 msgr2=0x7f981c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 --2- 172.21.15.19:0/1230749180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c100cd0 0x7f981c101150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f980c009900 tx=0x7f980c02a410 comp rx=0 tx=0).stop 2024-09-19T04:07:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 -- 172.21.15.19:0/1230749180 shutdown_connections 2024-09-19T04:07:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 --2- 172.21.15.19:0/1230749180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c101690 0x7f981c108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 --2- 172.21.15.19:0/1230749180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c100cd0 0x7f981c101150 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 --2- 172.21.15.19:0/1230749180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f981c102030 0x7f981c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 -- 172.21.15.19:0/1230749180 >> 172.21.15.19:0/1230749180 conn(0x7f981c0fb930 msgr2=0x7f981c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 -- 172.21.15.19:0/1230749180 shutdown_connections 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.509+0000 7f982407b640 1 -- 172.21.15.19:0/1230749180 wait complete. 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f982407b640 1 Processor -- start 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f982407b640 1 -- start start 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f982407b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f981c100cd0 0x7f981c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f982407b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c101690 0x7f981c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f982407b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c102030 0x7f981c19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f982407b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f981c10c6c0 con 0x7f981c101690 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f982407b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f981c10c540 con 0x7f981c100cd0 2024-09-19T04:07:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f982407b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f981c10c840 con 0x7f981c102030 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c101690 0x7f981c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c101690 0x7f981c1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57248/0 (socket says 172.21.15.19:57248) 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 -- 172.21.15.19:0/395056033 learned_addr learned my addr 172.21.15.19:0/395056033 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821e17640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f981c100cd0 0x7f981c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9822618640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c102030 0x7f981c19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 -- 172.21.15.19:0/395056033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c102030 msgr2=0x7f981c19f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c102030 0x7f981c19f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 -- 172.21.15.19:0/395056033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f981c100cd0 msgr2=0x7f981c197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f981c100cd0 0x7f981c197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 -- 172.21.15.19:0/395056033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c19f990 con 0x7f981c101690 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821e17640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f981c100cd0 0x7f981c197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.510+0000 7f9821616640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c101690 0x7f981c1982b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f980c009230 tx=0x7f980c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.511+0000 7f9812ffd640 1 -- 172.21.15.19:0/395056033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c03f030 con 0x7f981c101690 2024-09-19T04:07:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.511+0000 7f9812ffd640 1 -- 172.21.15.19:0/395056033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f980c003ce0 con 0x7f981c101690 2024-09-19T04:07:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.511+0000 7f982407b640 1 -- 172.21.15.19:0/395056033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981c19fc20 con 0x7f981c101690 2024-09-19T04:07:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.511+0000 7f9812ffd640 1 -- 172.21.15.19:0/395056033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c035070 con 0x7f981c101690 2024-09-19T04:07:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.511+0000 7f982407b640 1 -- 172.21.15.19:0/395056033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981c1a0110 con 0x7f981c101690 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9826e18740 1 Processor -- start 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9826e18740 1 -- start start 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9812ffd640 1 -- 172.21.15.19:0/395056033 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f980c042030 con 0x7f981c101690 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9826e18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d79a0 0x5605e62d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9826e18740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d82e0 0x5605e62dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9826e18740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62df830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9826e18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5605e623b2f0 con 0x5605e62d79a0 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9826e18740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5605e61e3270 con 0x5605e62dd370 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.513+0000 7f9826e18740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5605e62372b0 con 0x5605e62d82e0 2024-09-19T04:07:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d79a0 0x5605e62d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d79a0 0x5605e62d7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57262/0 (socket says 172.21.15.19:57262) 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 -- 172.21.15.19:0/1618190583 learned_addr learned my addr 172.21.15.19:0/1618190583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9821616640 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9821e17640 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d82e0 0x5605e62dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 -- 172.21.15.19:0/1618190583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d82e0 msgr2=0x5605e62dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d82e0 0x5605e62dcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 -- 172.21.15.19:0/1618190583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 msgr2=0x5605e62df830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62df830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 -- 172.21.15.19:0/1618190583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605e61e2340 con 0x5605e62d79a0 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9812ffd640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98080a3390 0x7f98080a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9822618640 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d79a0 0x5605e62d7da0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9814007e50 tx=0x7f981400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dca11d1038bd5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9812ffd640 1 -- 172.21.15.19:0/395056033 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f980c0f0550 con 0x7f981c101690 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9821e17640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98080a3390 0x7f98080a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9807fff640 1 -- 172.21.15.19:0/1618190583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814014070 con 0x5605e62d79a0 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9807fff640 1 -- 172.21.15.19:0/1618190583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981400ee90 con 0x5605e62d79a0 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.514+0000 7f9807fff640 1 -- 172.21.15.19:0/1618190583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98140054e0 con 0x5605e62d79a0 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 -- 172.21.15.19:0/1618190583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d79a0 msgr2=0x5605e62d7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9821e17640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98080a3390 0x7f98080a5850 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5605e62d8d10 tx=0x7f981800b570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d79a0 0x5605e62d7da0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9814007e50 tx=0x7f981400e720 comp rx=0 tx=0).stop 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 -- 172.21.15.19:0/1618190583 shutdown_connections 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62df830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d82e0 0x5605e62dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 --2- 172.21.15.19:0/1618190583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d79a0 0x5605e62d7da0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 -- 172.21.15.19:0/1618190583 >> 172.21.15.19:0/1618190583 conn(0x5605e62d3010 msgr2=0x5605e62d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 -- 172.21.15.19:0/1618190583 shutdown_connections 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.515+0000 7f9826e18740 1 -- 172.21.15.19:0/1618190583 wait complete. 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 1 -- start start 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 1 Processor -- start 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5605e62e0340 con 0x5605e62d82e0 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5605e61c4290 con 0x5605e62dd370 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5605e62e04c0 con 0x5605e62d79a0 2024-09-19T04:07:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9826e18740 10 client.0 _ll_register_callbacks cb 0x5605e61f5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9821616640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9821616640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57264/0 (socket says 172.21.15.19:57264) 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9821616640 1 -- 172.21.15.19:0/1087923438 learned_addr learned my addr 172.21.15.19:0/1087923438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9821e17640 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.516+0000 7f9822618640 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9821616640 1 -- 172.21.15.19:0/1087923438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 msgr2=0x5605e62e1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9821616640 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62e1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9821616640 1 -- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 msgr2=0x5605e62e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9821616640 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9821616640 1 -- 172.21.15.19:0/1087923438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605e5f4e210 con 0x5605e62d82e0 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9822618640 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9821616640 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e2160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f980c0b9040 tx=0x7f980c0b8310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721a159ad49a4d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f98057fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f98057fa640 1 -- 172.21.15.19:0/1087923438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c0ef070 con 0x5605e62d82e0 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f98057fa640 1 -- 172.21.15.19:0/1087923438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f980c0eddb0 con 0x5605e62d82e0 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 1 -- 172.21.15.19:0/1087923438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f981400ee90 con 0x5605e62d82e0 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f98057fa640 1 -- 172.21.15.19:0/1087923438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f980c049070 con 0x5605e62d82e0 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 10 client.15339 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 1 -- 172.21.15.19:0/1087923438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5605e601a620 con 0x5605e62d82e0 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f98057fa640 1 -- 172.21.15.19:0/1087923438 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981400ee90 con 0x5605e62d82e0 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f98057fa640 1 -- 172.21.15.19:0/1087923438 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f980c0b8ce0 con 0x5605e62d82e0 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 10 client.15339 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 20 client.15339 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5605e62ed950 0x5605e62efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 4 client.15339 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.517+0000 7f9826e18740 1 -- 172.21.15.19:0/1087923438 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605e6248a20 con 0x5605e62ed950 2024-09-19T04:07:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.518+0000 7f9821e17640 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5605e62ed950 0x5605e62efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.518+0000 7f9821e17640 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5605e62ed950 0x5605e62efdb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e6081406a0a216d server_cookie=3fc91a7958ae9859 in_seq=0 out_seq=0 2024-09-19T04:07:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.518+0000 7f98057fa640 10 client.15339 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f98057fa640 1 -- 172.21.15.19:0/1087923438 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605e6248a20 con 0x5605e62ed950 2024-09-19T04:07:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f98057fa640 10 client.15339 handle_command_reply: tid=0 2024-09-19T04:07:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 client.15339 shutdown 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 -- 172.21.15.19:0/1087923438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 msgr2=0x5605e62e2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e2160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f980c0b9040 tx=0x7f980c0b8310 comp rx=0 tx=0).stop 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 -- 172.21.15.19:0/1087923438 shutdown_connections 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e2160 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5605e62ed950 0x5605e62efdb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 --2- 172.21.15.19:0/1087923438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62e1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 -- 172.21.15.19:0/1087923438 >> 172.21.15.19:0/1087923438 conn(0x5605e62d3010 msgr2=0x5605e62d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f98057fa640 0 client.15339 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 -- 172.21.15.19:0/1087923438 shutdown_connections 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 1 -- 172.21.15.19:0/1087923438 wait complete. 2024-09-19T04:07:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.519+0000 7f9826e18740 20 client.15339 trim_cache size 0 max 16384 2024-09-19T04:07:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9826e18740 1 Processor -- start 2024-09-19T04:07:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9826e18740 1 -- start start 2024-09-19T04:07:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9826e18740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9826e18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9826e18740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9826e18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1aa6d0 con 0x5605e62d82e0 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9826e18740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1ac210 con 0x5605e62dd370 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9826e18740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1aada0 con 0x5605e62d79a0 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57280/0 (socket says 172.21.15.19:57280) 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 -- 172.21.15.19:0/199867068 learned_addr learned my addr 172.21.15.19:0/199867068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821e17640 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9822618640 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 -- 172.21.15.19:0/199867068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 msgr2=0x5605e62d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 -- 172.21.15.19:0/199867068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 msgr2=0x5605e62f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 -- 172.21.15.19:0/199867068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605e62f0430 con 0x5605e62d82e0 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821e17640 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9822618640 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.534+0000 7f9821616640 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e9160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f980c031440 tx=0x7f980c0f3c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f75d305030b426d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f98077fe640 1 -- 172.21.15.19:0/199867068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c0da070 con 0x5605e62d82e0 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f98077fe640 1 -- 172.21.15.19:0/199867068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f980c0044d0 con 0x5605e62d82e0 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f98077fe640 1 -- 172.21.15.19:0/199867068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c0ef050 con 0x5605e62d82e0 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 -- 172.21.15.19:0/199867068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 msgr2=0x5605e62e9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e9160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f980c031440 tx=0x7f980c0f3c20 comp rx=0 tx=0).stop 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 -- 172.21.15.19:0/199867068 shutdown_connections 2024-09-19T04:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62dd370 0x5605e62f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e9160 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 --2- 172.21.15.19:0/199867068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62d79a0 0x5605e62d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 -- 172.21.15.19:0/199867068 >> 172.21.15.19:0/199867068 conn(0x5605e62d3010 msgr2=0x5605e62d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 -- 172.21.15.19:0/199867068 shutdown_connections 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.535+0000 7f9826e18740 1 -- 172.21.15.19:0/199867068 wait complete. 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 1 -- start start 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 1 Processor -- start 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62d79a0 0x5605e62e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62dd370 0x5605e62e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1abb40 con 0x5605e62d82e0 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1ab470 con 0x5605e62d79a0 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1a86b0 con 0x5605e62dd370 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9826e18740 10 client.0 _ll_register_callbacks cb 0x7f980c0f3d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9821616640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9821616640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57284/0 (socket says 172.21.15.19:57284) 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9821616640 1 -- 172.21.15.19:0/3867845634 learned_addr learned my addr 172.21.15.19:0/3867845634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9821e17640 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62d79a0 0x5605e62e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9821616640 1 -- 172.21.15.19:0/3867845634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62dd370 msgr2=0x5605e62e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9822618640 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62dd370 0x5605e62e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.536+0000 7f9821616640 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62dd370 0x5605e62e8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9821616640 1 -- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62d79a0 msgr2=0x5605e62e6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9821616640 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62d79a0 0x5605e62e6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9821616640 1 -- 172.21.15.19:0/3867845634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605e62e8750 con 0x5605e62d82e0 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9821e17640 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62d79a0 0x5605e62e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9821616640 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e7130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f980c0bc370 tx=0x7f980c0058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4849779e80c720 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f98057fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f98057fa640 1 -- 172.21.15.19:0/3867845634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c086060 con 0x5605e62d82e0 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f98057fa640 1 -- 172.21.15.19:0/3867845634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f980c0f0260 con 0x5605e62d82e0 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 1 -- 172.21.15.19:0/3867845634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f980c0044d0 con 0x5605e62d82e0 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f98057fa640 1 -- 172.21.15.19:0/3867845634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f980c0da070 con 0x5605e62d82e0 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 10 client.15345 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 1 -- 172.21.15.19:0/3867845634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5605e62e8750 con 0x5605e62d82e0 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f98057fa640 1 -- 172.21.15.19:0/3867845634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f980c0044d0 con 0x5605e62d82e0 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f98057fa640 1 -- 172.21.15.19:0/3867845634 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f980c09d030 con 0x5605e62d82e0 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 10 client.15345 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 20 client.15345 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5605e6358f30 0x5605e635b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 4 client.15345 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.537+0000 7f9826e18740 1 -- 172.21.15.19:0/3867845634 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605e6242d70 con 0x5605e6358f30 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.538+0000 7f9821e17640 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5605e6358f30 0x5605e635b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.538+0000 7f9821e17640 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5605e6358f30 0x5605e635b350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b99f5aa4337a0c0 server_cookie=fd059b51821d6fbb in_seq=0 out_seq=0 2024-09-19T04:07:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.538+0000 7f98057fa640 10 client.15345 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.551+0000 7f98057fa640 1 -- 172.21.15.19:0/3867845634 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605e6242d70 con 0x5605e6358f30 2024-09-19T04:07:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.551+0000 7f98057fa640 10 client.15345 handle_command_reply: tid=0 2024-09-19T04:07:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.551+0000 7f9826e18740 1 client.15345 shutdown 2024-09-19T04:07:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 -- 172.21.15.19:0/3867845634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 msgr2=0x5605e62e7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e7130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f980c0bc370 tx=0x7f980c0058f0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 -- 172.21.15.19:0/3867845634 shutdown_connections 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5605e6358f30 0x5605e635b350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605e62dd370 0x5605e62e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605e62d82e0 0x5605e62e7130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f98057fa640 0 client.15345 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 --2- 172.21.15.19:0/3867845634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605e62d79a0 0x5605e62e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 -- 172.21.15.19:0/3867845634 >> 172.21.15.19:0/3867845634 conn(0x5605e62d3010 msgr2=0x5605e62d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 -- 172.21.15.19:0/3867845634 shutdown_connections 2024-09-19T04:07:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 1 -- 172.21.15.19:0/3867845634 wait complete. 2024-09-19T04:07:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f9826e18740 20 client.15345 trim_cache size 0 max 16384 2024-09-19T04:07:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:07:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:07:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:07:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f982407b640 1 -- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98080a3390 msgr2=0x7f98080a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f982407b640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98080a3390 0x7f98080a5850 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5605e62d8d10 tx=0x7f981800b570 comp rx=0 tx=0).stop 2024-09-19T04:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f982407b640 1 -- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c101690 msgr2=0x7f981c1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.552+0000 7f982407b640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c101690 0x7f981c1982b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f980c009230 tx=0x7f980c004360 comp rx=0 tx=0).stop 2024-09-19T04:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.553+0000 7f982407b640 1 -- 172.21.15.19:0/395056033 shutdown_connections 2024-09-19T04:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.553+0000 7f982407b640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f98080a3390 0x7f98080a5850 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.553+0000 7f982407b640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c102030 0x7f981c19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.553+0000 7f982407b640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f981c101690 0x7f981c1982b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.553+0000 7f982407b640 1 --2- 172.21.15.19:0/395056033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f981c100cd0 0x7f981c197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.553+0000 7f982407b640 1 -- 172.21.15.19:0/395056033 >> 172.21.15.19:0/395056033 conn(0x7f981c0fb930 msgr2=0x7f981c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.553+0000 7f982407b640 1 -- 172.21.15.19:0/395056033 shutdown_connections 2024-09-19T04:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:17.553+0000 7f982407b640 1 -- 172.21.15.19:0/395056033 wait complete. 2024-09-19T04:07:17.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T04:04:40.077653+0000', 'modified': '2024-09-19T04:07:03.493755+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24469, 'mds_2': 24454, 'mds_3': 24452, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14574': {'gid': 14574, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/821554910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 821554910}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 821554910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14580': {'gid': 14580, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/733319067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 733319067}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 733319067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6833/3618359356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 3618359356}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 3618359356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/2394579385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 2394579385}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 2394579385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/3717777626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 3717777626}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 3717777626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.196:6835/1584544131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1584544131}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1584544131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.111:6837/1580295791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 1580295791}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 1580295791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.196:6837/1041250110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1041250110}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1041250110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.196:6839/1173261295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1173261295}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1173261295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.111:6839/528342006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 528342006}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 528342006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24448, 24452, 24454, 24469, 24476]} 2024-09-19T04:07:17.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T04:07:17.923 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T04:07:18.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:17 smithi019 ceph-mon[30938]: pgmap v371: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T04:07:18.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:17 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1737673868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:18.220 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:18.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:17 smithi111 ceph-mon[35365]: pgmap v371: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T04:07:18.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:17 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1737673868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:17 smithi196 ceph-mon[35731]: pgmap v371: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T04:07:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:17 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1737673868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 -- 172.21.15.19:0/4160681036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8102790 msgr2=0x7fa5a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/4160681036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8102790 0x7fa5a81067e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009900 tx=0x7fa59c02a410 comp rx=0 tx=0).stop 2024-09-19T04:07:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 -- 172.21.15.19:0/4160681036 shutdown_connections 2024-09-19T04:07:18.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/4160681036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8106f10 0x7fa5a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/4160681036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8102790 0x7fa5a81067e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/4160681036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8101de0 0x7fa5a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 -- 172.21.15.19:0/4160681036 >> 172.21.15.19:0/4160681036 conn(0x7fa5a80fd8c0 msgr2=0x7fa5a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:18.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 -- 172.21.15.19:0/4160681036 shutdown_connections 2024-09-19T04:07:18.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.841+0000 7fa5afcf6640 1 -- 172.21.15.19:0/4160681036 wait complete. 2024-09-19T04:07:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5afcf6640 1 Processor -- start 2024-09-19T04:07:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5afcf6640 1 -- start start 2024-09-19T04:07:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5afcf6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8101de0 0x7fa5a8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5afcf6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8102790 0x7fa5a81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5afcf6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8106f10 0x7fa5a819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5afcf6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8111fc0 con 0x7fa5a8106f10 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5afcf6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8111e40 con 0x7fa5a8101de0 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5afcf6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8112140 con 0x7fa5a8102790 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5ae293640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8106f10 0x7fa5a819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5ae293640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8106f10 0x7fa5a819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57298/0 (socket says 172.21.15.19:57298) 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5ae293640 1 -- 172.21.15.19:0/3819614050 learned_addr learned my addr 172.21.15.19:0/3819614050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5ada92640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8101de0 0x7fa5a8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.842+0000 7fa5ad291640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8102790 0x7fa5a81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5ae293640 1 -- 172.21.15.19:0/3819614050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8102790 msgr2=0x7fa5a81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5ae293640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8102790 0x7fa5a81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5ae293640 1 -- 172.21.15.19:0/3819614050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8101de0 msgr2=0x7fa5a8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5ae293640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8101de0 0x7fa5a8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5ae293640 1 -- 172.21.15.19:0/3819614050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a819d410 con 0x7fa5a8106f10 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5ae293640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8106f10 0x7fa5a819cd30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa598009fd0 tx=0x7fa59800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5a6ffd640 1 -- 172.21.15.19:0/3819614050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598017070 con 0x7fa5a8106f10 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5a6ffd640 1 -- 172.21.15.19:0/3819614050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa598003e50 con 0x7fa5a8106f10 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5a6ffd640 1 -- 172.21.15.19:0/3819614050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980106c0 con 0x7fa5a8106f10 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5afcf6640 1 -- 172.21.15.19:0/3819614050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a819d700 con 0x7fa5a8106f10 2024-09-19T04:07:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.843+0000 7fa5afcf6640 1 -- 172.21.15.19:0/3819614050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a8074420 con 0x7fa5a8106f10 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.845+0000 7fa5a6ffd640 1 -- 172.21.15.19:0/3819614050 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fa598003970 con 0x7fa5a8106f10 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5b20cf740 1 Processor -- start 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5b20cf740 1 -- start start 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a96e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599a963e090 con 0x5599a96d82f0 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5599a95e5a70 con 0x5599a96e2bb0 2024-09-19T04:07:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599a963a050 con 0x5599a96d79d0 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5a6ffd640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa5900a3350 0x7fa5900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5ada92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5ada92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57314/0 (socket says 172.21.15.19:57314) 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5ada92640 1 -- 172.21.15.19:0/1771122703 learned_addr learned my addr 172.21.15.19:0/1771122703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5ad291640 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5ae293640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa5900a3350 0x7fa5900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.846+0000 7fa5ad291640 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a96e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5a6ffd640 1 -- 172.21.15.19:0/3819614050 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fa5980cfa30 con 0x7fa5a8106f10 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5ada92640 1 -- 172.21.15.19:0/1771122703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 msgr2=0x5599a96d7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5ada92640 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96d7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5ada92640 1 -- 172.21.15.19:0/1771122703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 msgr2=0x5599a96e4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:18.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5ada92640 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a96e4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5ada92640 1 -- 172.21.15.19:0/1771122703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a95f2130 con 0x5599a96d82f0 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5ad291640 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96d7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5ada92640 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96e2670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000b760 tx=0x7fa5a000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6ec6e1e7cd8f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa5ae293640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa5900a3350 0x7fa5900a5810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa59800b760 tx=0x7fa598018fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa58ffff640 1 -- 172.21.15.19:0/1771122703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0003dd0 con 0x5599a96d82f0 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa58ffff640 1 -- 172.21.15.19:0/1771122703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0015070 con 0x5599a96d82f0 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.847+0000 7fa58ffff640 1 -- 172.21.15.19:0/1771122703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0011440 con 0x5599a96d82f0 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 -- 172.21.15.19:0/1771122703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 msgr2=0x5599a96e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96e2670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000b760 tx=0x7fa5a000bc20 comp rx=0 tx=0).stop 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 -- 172.21.15.19:0/1771122703 shutdown_connections 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a96e4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96e2670 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/1771122703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96d7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 -- 172.21.15.19:0/1771122703 >> 172.21.15.19:0/1771122703 conn(0x5599a96d3f60 msgr2=0x5599a96d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 -- 172.21.15.19:0/1771122703 shutdown_connections 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 -- 172.21.15.19:0/1771122703 wait complete. 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.848+0000 7fa5b20cf740 1 -- start start 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5b20cf740 1 Processor -- start 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96e2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96d82f0 0x5599a96df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96e2bb0 0x5599a96dfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599a96e5780 con 0x5599a96e2bb0 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5599a95f8f20 con 0x5599a96d82f0 2024-09-19T04:07:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599a96e5900 con 0x5599a96d79d0 2024-09-19T04:07:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5b20cf740 10 client.0 _ll_register_callbacks cb 0x5599a95c2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5ad291640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96e2bb0 0x5599a96dfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5ad291640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96e2bb0 0x5599a96dfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57322/0 (socket says 172.21.15.19:57322) 2024-09-19T04:07:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5ad291640 1 -- 172.21.15.19:0/4014133417 learned_addr learned my addr 172.21.15.19:0/4014133417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5ada92640 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96d82f0 0x5599a96df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.849+0000 7fa5ad291640 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5ada92640 1 -- 172.21.15.19:0/4014133417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 msgr2=0x5599a96e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5ada92640 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5ada92640 1 -- 172.21.15.19:0/4014133417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96e2bb0 msgr2=0x5599a96dfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5ada92640 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96e2bb0 0x5599a96dfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5ada92640 1 -- 172.21.15.19:0/4014133417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a93af2b0 con 0x5599a96d82f0 2024-09-19T04:07:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5ad291640 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96e2bb0 0x5599a96dfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5ada92640 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96d82f0 0x5599a96df7d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00039c0 tx=0x7fa5a00048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742ad91f017f5f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa58d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-19T04:07:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa58d7fa640 1 -- 172.21.15.19:0/4014133417 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0028060 con 0x5599a96d82f0 2024-09-19T04:07:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5b20cf740 1 -- 172.21.15.19:0/4014133417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a0015070 con 0x5599a96d82f0 2024-09-19T04:07:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa58d7fa640 1 -- 172.21.15.19:0/4014133417 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fa5a0011640 con 0x5599a96d82f0 2024-09-19T04:07:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa58d7fa640 1 -- 172.21.15.19:0/4014133417 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5a0003ea0 con 0x5599a96d82f0 2024-09-19T04:07:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5b20cf740 10 client.25007 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa5b20cf740 1 -- 172.21.15.19:0/4014133417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599a96f1130 con 0x5599a96d82f0 2024-09-19T04:07:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.850+0000 7fa58d7fa640 1 -- 172.21.15.19:0/4014133417 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0015070 con 0x5599a96d82f0 2024-09-19T04:07:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa58d7fa640 1 -- 172.21.15.19:0/4014133417 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa5a0022070 con 0x5599a96d82f0 2024-09-19T04:07:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa5b20cf740 10 client.25007 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa5b20cf740 10 client.25007 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:07:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa5b20cf740 20 client.25007 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x5599a96f33f0 0x5599a96f4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa5b20cf740 4 client.25007 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa5b20cf740 1 -- 172.21.15.19:0/4014133417 --> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599a964b600 con 0x5599a96f33f0 2024-09-19T04:07:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa5ad291640 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x5599a96f33f0 0x5599a96f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa5ad291640 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x5599a96f33f0 0x5599a96f4860 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=64e11803585c2996 server_cookie=bc62fa071a90eb19 in_seq=0 out_seq=0 2024-09-19T04:07:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.851+0000 7fa58d7fa640 10 client.25007 ms_handle_connect on v2:172.21.15.196:6834/1584544131 2024-09-19T04:07:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.852+0000 7fa58d7fa640 1 -- 172.21.15.19:0/4014133417 <== mds.4 v2:172.21.15.196:6834/1584544131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599a964b600 con 0x5599a96f33f0 2024-09-19T04:07:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.852+0000 7fa58d7fa640 10 client.25007 handle_command_reply: tid=0 2024-09-19T04:07:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.852+0000 7fa5b20cf740 1 client.25007 shutdown 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.852+0000 7fa5b20cf740 1 -- 172.21.15.19:0/4014133417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96d82f0 msgr2=0x5599a96df7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.852+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96d82f0 0x5599a96df7d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00039c0 tx=0x7fa5a00048f0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 1 -- 172.21.15.19:0/4014133417 shutdown_connections 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x5599a96f33f0 0x5599a96f4860 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96e2bb0 0x5599a96dfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96d82f0 0x5599a96df7d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa58d7fa640 0 client.25007 ms_handle_reset on v2:172.21.15.196:6834/1584544131 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/4014133417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 1 -- 172.21.15.19:0/4014133417 >> 172.21.15.19:0/4014133417 conn(0x5599a96d3f60 msgr2=0x5599a96d9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 1 -- 172.21.15.19:0/4014133417 shutdown_connections 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 1 -- 172.21.15.19:0/4014133417 wait complete. 2024-09-19T04:07:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.853+0000 7fa5b20cf740 20 client.25007 trim_cache size 0 max 16384 2024-09-19T04:07:18.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.870+0000 7fa5b20cf740 1 Processor -- start 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.870+0000 7fa5b20cf740 1 -- start start 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a96dca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599a96f2ba0 con 0x5599a96d82f0 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5599a96f3270 con 0x5599a96e2bb0 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599a96f3940 con 0x5599a96d79d0 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57330/0 (socket says 172.21.15.19:57330) 2024-09-19T04:07:18.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 -- 172.21.15.19:0/3954144662 learned_addr learned my addr 172.21.15.19:0/3954144662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ad291640 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 -- 172.21.15.19:0/3954144662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 msgr2=0x5599a96f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 -- 172.21.15.19:0/3954144662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 msgr2=0x5599a96dca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a96dca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 -- 172.21.15.19:0/3954144662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a96d8120 con 0x5599a96d82f0 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ad291640 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa5ada92640 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96df090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000b760 tx=0x7fa5a0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef9c457a3acdf66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa58f7fe640 1 -- 172.21.15.19:0/3954144662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00055a0 con 0x5599a96d82f0 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa58f7fe640 1 -- 172.21.15.19:0/3954144662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0015040 con 0x5599a96d82f0 2024-09-19T04:07:18.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.871+0000 7fa58f7fe640 1 -- 172.21.15.19:0/3954144662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0011ad0 con 0x5599a96d82f0 2024-09-19T04:07:18.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 -- 172.21.15.19:0/3954144662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 msgr2=0x5599a96df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96df090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000b760 tx=0x7fa5a0005f30 comp rx=0 tx=0).stop 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 -- 172.21.15.19:0/3954144662 shutdown_connections 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a96dca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d82f0 0x5599a96df090 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/3954144662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d79d0 0x5599a96f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 -- 172.21.15.19:0/3954144662 >> 172.21.15.19:0/3954144662 conn(0x5599a96d3f60 msgr2=0x5599a96de0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 -- 172.21.15.19:0/3954144662 shutdown_connections 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 -- 172.21.15.19:0/3954144662 wait complete. 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 -- start start 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.872+0000 7fa5b20cf740 1 Processor -- start 2024-09-19T04:07:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d79d0 0x5599a96f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d82f0 0x5599a96f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5b20cf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a975baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599a96f46e0 con 0x5599a96d79d0 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5599a96f4010 con 0x5599a96e2bb0 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5b20cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599a96f4db0 con 0x5599a96d82f0 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5b20cf740 10 client.0 _ll_register_callbacks cb 0x7fa5a0004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ad291640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d79d0 0x5599a96f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ad291640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d79d0 0x5599a96f7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57344/0 (socket says 172.21.15.19:57344) 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ad291640 1 -- 172.21.15.19:0/2248508811 learned_addr learned my addr 172.21.15.19:0/2248508811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ada92640 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d82f0 0x5599a96f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ad291640 1 -- 172.21.15.19:0/2248508811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d82f0 msgr2=0x5599a96f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ad291640 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d82f0 0x5599a96f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ad291640 1 -- 172.21.15.19:0/2248508811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 msgr2=0x5599a975baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ad291640 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a975baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.873+0000 7fa5ad291640 1 -- 172.21.15.19:0/2248508811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a96f8a30 con 0x5599a96d79d0 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa5ad291640 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d79d0 0x5599a96f7c00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa59c02a930 tx=0x7fa59c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe87177fb4172238 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa58d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa58d7fa640 1 -- 172.21.15.19:0/2248508811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c04f060 con 0x5599a96d79d0 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa58d7fa640 1 -- 172.21.15.19:0/2248508811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fa59c04be80 con 0x5599a96d79d0 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa5b20cf740 1 -- 172.21.15.19:0/2248508811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a0015040 con 0x5599a96d79d0 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa58d7fa640 1 -- 172.21.15.19:0/2248508811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa59c035040 con 0x5599a96d79d0 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa5b20cf740 10 client.15363 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa5b20cf740 1 -- 172.21.15.19:0/2248508811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599a975d130 con 0x5599a96d79d0 2024-09-19T04:07:18.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa58d7fa640 1 -- 172.21.15.19:0/2248508811 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0015040 con 0x5599a96d79d0 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa58d7fa640 1 -- 172.21.15.19:0/2248508811 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa59c0315b0 con 0x5599a96d79d0 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.874+0000 7fa5b20cf740 10 client.15363 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.875+0000 7fa5b20cf740 10 client.15363 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.875+0000 7fa5b20cf740 20 client.15363 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.875+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x5599a9761c60 0x5599a9760970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.875+0000 7fa5b20cf740 4 client.15363 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.875+0000 7fa5b20cf740 1 -- 172.21.15.19:0/2248508811 --> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] -- command(tid 0: {"prefix": "status"}) -- 0x5599a9645b10 con 0x5599a9761c60 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.875+0000 7fa5ada92640 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x5599a9761c60 0x5599a9760970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.875+0000 7fa58d7fa640 10 client.15363 ms_handle_connect on v2:172.21.15.196:6834/1584544131 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.875+0000 7fa5ada92640 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x5599a9761c60 0x5599a9760970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e649738a277fd6f server_cookie=8c76bd2a5d21c969 in_seq=0 out_seq=0 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa58d7fa640 1 -- 172.21.15.19:0/2248508811 <== mds.4 v2:172.21.15.196:6834/1584544131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5599a9645b10 con 0x5599a9761c60 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa58d7fa640 10 client.15363 handle_command_reply: tid=0 2024-09-19T04:07:18.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 client.15363 shutdown 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 -- 172.21.15.19:0/2248508811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d79d0 msgr2=0x5599a96f7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d79d0 0x5599a96f7c00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa59c02a930 tx=0x7fa59c005b70 comp rx=0 tx=0).stop 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 -- 172.21.15.19:0/2248508811 shutdown_connections 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a96e2bb0 0x5599a975baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599a96d82f0 0x5599a96f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.196:6834/1584544131,v1:172.21.15.196:6835/1584544131] conn(0x5599a9761c60 0x5599a9760970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 --2- 172.21.15.19:0/2248508811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599a96d79d0 0x5599a96f7c00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa58d7fa640 0 client.15363 ms_handle_reset on v2:172.21.15.196:6834/1584544131 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 -- 172.21.15.19:0/2248508811 >> 172.21.15.19:0/2248508811 conn(0x5599a96d3f60 msgr2=0x5599a96d2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 -- 172.21.15.19:0/2248508811 shutdown_connections 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 1 -- 172.21.15.19:0/2248508811 wait complete. 2024-09-19T04:07:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.876+0000 7fa5b20cf740 20 client.15363 trim_cache size 0 max 16384 2024-09-19T04:07:18.884 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:07:18.884 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:07:18.884 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-09-19T04:07:18.884 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24448, 2024-09-19T04:07:18.884 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:07:18.884 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:07:18.884 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:07:18.885 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 116.355724542, 2024-09-19T04:07:18.885 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 29, 2024-09-19T04:07:18.885 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:07:18.885 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 93, 2024-09-19T04:07:18.885 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 185.49631967799999 2024-09-19T04:07:18.885 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 -- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa5900a3350 msgr2=0x7fa5900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa5900a3350 0x7fa5900a5810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa59800b760 tx=0x7fa598018fb0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 -- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8106f10 msgr2=0x7fa5a819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8106f10 0x7fa5a819cd30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa598009fd0 tx=0x7fa59800b260 comp rx=0 tx=0).stop 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 -- 172.21.15.19:0/3819614050 shutdown_connections 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa5900a3350 0x7fa5900a5810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a8106f10 0x7fa5a819cd30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8102790 0x7fa5a81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 --2- 172.21.15.19:0/3819614050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8101de0 0x7fa5a8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 -- 172.21.15.19:0/3819614050 >> 172.21.15.19:0/3819614050 conn(0x7fa5a80fd8c0 msgr2=0x7fa5a80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 -- 172.21.15.19:0/3819614050 shutdown_connections 2024-09-19T04:07:18.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:18.877+0000 7fa5afcf6640 1 -- 172.21.15.19:0/3819614050 wait complete. 2024-09-19T04:07:19.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T04:07:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:20.140 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:19 smithi019 ceph-mon[30938]: pgmap v372: 129 pgs: 129 active+clean; 664 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-19T04:07:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.349+0000 7f164e23d640 1 -- 172.21.15.19:0/1849404440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648102ef0 msgr2=0x7f164810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.350+0000 7f164e23d640 1 --2- 172.21.15.19:0/1849404440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648102ef0 0x7f164810da40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1640009ca0 tx=0x7f164002a950 comp rx=0 tx=0).stop 2024-09-19T04:07:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.350+0000 7f164e23d640 1 -- 172.21.15.19:0/1849404440 shutdown_connections 2024-09-19T04:07:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.350+0000 7f164e23d640 1 --2- 172.21.15.19:0/1849404440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648102ef0 0x7f164810da40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.350+0000 7f164e23d640 1 --2- 172.21.15.19:0/1849404440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1648102570 0x7f16481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.350+0000 7f164e23d640 1 --2- 172.21.15.19:0/1849404440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1648100e20 0x7f1648101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.350+0000 7f164e23d640 1 -- 172.21.15.19:0/1849404440 >> 172.21.15.19:0/1849404440 conn(0x7f16480fc7d0 msgr2=0x7f16480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.350+0000 7f164e23d640 1 -- 172.21.15.19:0/1849404440 shutdown_connections 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.350+0000 7f164e23d640 1 -- 172.21.15.19:0/1849404440 wait complete. 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f164e23d640 1 Processor -- start 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f164e23d640 1 -- start start 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f164e23d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e20 0x7f16480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f164e23d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1648102570 0x7f16480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f164e23d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1648102ef0 0x7f16480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f164e23d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1648110f80 con 0x7f1648100e20 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f164e23d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1648110e00 con 0x7f1648102ef0 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f164e23d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1648111100 con 0x7f1648102570 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f16477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e20 0x7f16480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f16477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e20 0x7f16480fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57362/0 (socket says 172.21.15.19:57362) 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f16477fe640 1 -- 172.21.15.19:0/3353713902 learned_addr learned my addr 172.21.15.19:0/3353713902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f163ffff640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1648102570 0x7f16480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.351+0000 7f1647fff640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1648102ef0 0x7f16480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16477fe640 1 -- 172.21.15.19:0/3353713902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1648102570 msgr2=0x7f16480fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16477fe640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1648102570 0x7f16480fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16477fe640 1 -- 172.21.15.19:0/3353713902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1648102ef0 msgr2=0x7f16480fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16477fe640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1648102ef0 0x7f16480fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16477fe640 1 -- 172.21.15.19:0/3353713902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16481aada0 con 0x7f1648100e20 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f163ffff640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1648102570 0x7f16480fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f1647fff640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1648102ef0 0x7f16480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16477fe640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e20 0x7f16480fcc80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f163800c920 tx=0x7f163800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16457fa640 1 -- 172.21.15.19:0/3353713902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638007c70 con 0x7f1648100e20 2024-09-19T04:07:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16457fa640 1 -- 172.21.15.19:0/3353713902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1638007e10 con 0x7f1648100e20 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f16457fa640 1 -- 172.21.15.19:0/3353713902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638018890 con 0x7f1648100e20 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f164e23d640 1 -- 172.21.15.19:0/3353713902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16481aafd0 con 0x7f1648100e20 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.352+0000 7f164e23d640 1 -- 172.21.15.19:0/3353713902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16481ab480 con 0x7f1648100e20 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f16457fa640 1 -- 172.21.15.19:0/3353713902 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f1638018a70 con 0x7f1648100e20 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1650616740 1 Processor -- start 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1650616740 1 -- start start 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1650616740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabd0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1650616740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabdb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1650616740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1650616740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ffab37090 con 0x564ffabd12f0 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1650616740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564ffaadea70 con 0x564ffabdbbb0 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1650616740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ffab33050 con 0x564ffabd09d0 2024-09-19T04:07:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f16457fa640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16340a3350 0x7f16340a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f16477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabdb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f16477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabdb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57370/0 (socket says 172.21.15.19:57370) 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f16477fe640 1 -- 172.21.15.19:0/559780169 learned_addr learned my addr 172.21.15.19:0/559780169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.355+0000 7f1647fff640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16340a3350 0x7f16340a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163ffff640 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f1647fff640 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabd0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f16457fa640 1 -- 172.21.15.19:0/3353713902 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f16380cff10 con 0x7f1648100e20 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163ffff640 1 -- 172.21.15.19:0/559780169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 msgr2=0x564ffabd0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163ffff640 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabd0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163ffff640 1 -- 172.21.15.19:0/559780169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 msgr2=0x564ffabdb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163ffff640 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabdb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163ffff640 1 -- 172.21.15.19:0/559780169 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ffaaeb130 con 0x564ffabdbbb0 2024-09-19T04:07:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f16477fe640 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabdb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f1647fff640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16340a3350 0x7f16340a5810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1640005e30 tx=0x7f1640009880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163ffff640 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabddfa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16480fe4d0 tx=0x7f16300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7653ecfc26893676 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163dffb640 1 -- 172.21.15.19:0/559780169 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300040f0 con 0x564ffabdbbb0 2024-09-19T04:07:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163dffb640 1 -- 172.21.15.19:0/559780169 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630005500 con 0x564ffabdbbb0 2024-09-19T04:07:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.356+0000 7f163dffb640 1 -- 172.21.15.19:0/559780169 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300035f0 con 0x564ffabdbbb0 2024-09-19T04:07:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 -- 172.21.15.19:0/559780169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 msgr2=0x564ffabddfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabddfa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16480fe4d0 tx=0x7f16300118d0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 -- 172.21.15.19:0/559780169 shutdown_connections 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabddfa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabdb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 --2- 172.21.15.19:0/559780169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabd0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 -- 172.21.15.19:0/559780169 >> 172.21.15.19:0/559780169 conn(0x564ffabccf60 msgr2=0x564ffabcd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 -- 172.21.15.19:0/559780169 shutdown_connections 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 -- 172.21.15.19:0/559780169 wait complete. 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 -- start start 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.358+0000 7f1650616740 1 Processor -- start 2024-09-19T04:07:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f1650616740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd09d0 0x564ffabdb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f1650616740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabd8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f1650616740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabdbbb0 0x564ffabd8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f1650616740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ffabde780 con 0x564ffabd12f0 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f1650616740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564ffaaf1f20 con 0x564ffabd09d0 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f1650616740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ffabde900 con 0x564ffabdbbb0 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f1650616740 10 client.0 _ll_register_callbacks cb 0x564ffaabb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f163ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabd8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f163ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabd8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57378/0 (socket says 172.21.15.19:57378) 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f163ffff640 1 -- 172.21.15.19:0/3005579312 learned_addr learned my addr 172.21.15.19:0/3005579312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f16477fe640 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd09d0 0x564ffabdb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f1647fff640 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabdbbb0 0x564ffabd8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f163ffff640 1 -- 172.21.15.19:0/3005579312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabdbbb0 msgr2=0x564ffabd8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f163ffff640 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabdbbb0 0x564ffabd8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f163ffff640 1 -- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd09d0 msgr2=0x564ffabdb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f163ffff640 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd09d0 0x564ffabdb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f163ffff640 1 -- 172.21.15.19:0/3005579312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ffa8a82b0 con 0x564ffabd12f0 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.359+0000 7f16477fe640 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd09d0 0x564ffabdb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f163ffff640 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabd8790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f163001b040 tx=0x7f1630004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2996b5a024820e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f1626ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f1626ffd640 1 -- 172.21.15.19:0/3005579312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630029020 con 0x564ffabd12f0 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f1626ffd640 1 -- 172.21.15.19:0/3005579312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f16300051c0 con 0x564ffabd12f0 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f1650616740 1 -- 172.21.15.19:0/3005579312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1630005500 con 0x564ffabd12f0 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f1626ffd640 1 -- 172.21.15.19:0/3005579312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1630034050 con 0x564ffabd12f0 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f1650616740 10 client.15375 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f1650616740 1 -- 172.21.15.19:0/3005579312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ffabea090 con 0x564ffabd12f0 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.360+0000 7f1626ffd640 1 -- 172.21.15.19:0/3005579312 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630005500 con 0x564ffabd12f0 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f1626ffd640 1 -- 172.21.15.19:0/3005579312 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1630025470 con 0x564ffabd12f0 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f1650616740 10 client.15375 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f1650616740 10 client.15375 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f1650616740 20 client.15375 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f1650616740 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x564ffabecac0 0x564ffabedf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f1650616740 4 client.15375 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f1650616740 1 -- 172.21.15.19:0/3005579312 --> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ffab44600 con 0x564ffabecac0 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f16477fe640 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x564ffabecac0 0x564ffabedf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f16477fe640 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x564ffabecac0 0x564ffabedf30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a645bfaa41401ce2 server_cookie=b61a7cd6b18ce322 in_seq=0 out_seq=0 2024-09-19T04:07:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.361+0000 7f1626ffd640 10 client.15375 ms_handle_connect on v2:172.21.15.111:6836/1580295791 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1626ffd640 1 -- 172.21.15.19:0/3005579312 <== mds.3 v2:172.21.15.111:6836/1580295791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ffab44600 con 0x564ffabecac0 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1626ffd640 10 client.15375 handle_command_reply: tid=0 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 client.15375 shutdown 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 -- 172.21.15.19:0/3005579312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 msgr2=0x564ffabd8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabd8790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f163001b040 tx=0x7f1630004e10 comp rx=0 tx=0).stop 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 -- 172.21.15.19:0/3005579312 shutdown_connections 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x564ffabecac0 0x564ffabedf30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabdbbb0 0x564ffabd8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd12f0 0x564ffabd8790 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 --2- 172.21.15.19:0/3005579312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd09d0 0x564ffabdb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 -- 172.21.15.19:0/3005579312 >> 172.21.15.19:0/3005579312 conn(0x564ffabccf60 msgr2=0x564ffabd2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1626ffd640 0 client.15375 ms_handle_reset on v2:172.21.15.111:6836/1580295791 2024-09-19T04:07:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 -- 172.21.15.19:0/3005579312 shutdown_connections 2024-09-19T04:07:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.362+0000 7f1650616740 1 -- 172.21.15.19:0/3005579312 wait complete. 2024-09-19T04:07:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.363+0000 7f1650616740 20 client.15375 trim_cache size 0 max 16384 2024-09-19T04:07:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.393+0000 7f1650616740 1 Processor -- start 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.393+0000 7f1650616740 1 -- start start 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f1650616740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabedf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f1650616740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd12f0 0x564ffabebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f1650616740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabdbbb0 0x564ffabd3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f1650616740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ffabec270 con 0x564ffabdbbb0 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f1650616740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564ffabec940 con 0x564ffabd12f0 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f1650616740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ffabed010 con 0x564ffabd09d0 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f16477fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabedf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f16477fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabedf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:41242/0 (socket says 172.21.15.19:41242) 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f16477fe640 1 -- 172.21.15.19:0/294740077 learned_addr learned my addr 172.21.15.19:0/294740077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f163ffff640 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd12f0 0x564ffabebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f1647fff640 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabdbbb0 0x564ffabd3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f16477fe640 1 -- 172.21.15.19:0/294740077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd12f0 msgr2=0x564ffabebd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f16477fe640 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd12f0 0x564ffabebd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f16477fe640 1 -- 172.21.15.19:0/294740077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabdbbb0 msgr2=0x564ffabd3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f16477fe640 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabdbbb0 0x564ffabd3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.394+0000 7f16477fe640 1 -- 172.21.15.19:0/294740077 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ffabcca00 con 0x564ffabd09d0 2024-09-19T04:07:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.395+0000 7f1647fff640 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabdbbb0 0x564ffabd3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.395+0000 7f16477fe640 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabedf30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f163800c8f0 tx=0x7f16380bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3634155451f922d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.395+0000 7f163d7fa640 1 -- 172.21.15.19:0/294740077 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16380a4ca0 con 0x564ffabd09d0 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.395+0000 7f163d7fa640 1 -- 172.21.15.19:0/294740077 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163809a0d0 con 0x564ffabd09d0 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.395+0000 7f163d7fa640 1 -- 172.21.15.19:0/294740077 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163800fb30 con 0x564ffabd09d0 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 -- 172.21.15.19:0/294740077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 msgr2=0x564ffabedf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabedf30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f163800c8f0 tx=0x7f16380bc040 comp rx=0 tx=0).stop 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 -- 172.21.15.19:0/294740077 shutdown_connections 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabdbbb0 0x564ffabd3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabd12f0 0x564ffabebd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 --2- 172.21.15.19:0/294740077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd09d0 0x564ffabedf30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 -- 172.21.15.19:0/294740077 >> 172.21.15.19:0/294740077 conn(0x564ffabccf60 msgr2=0x564ffabd3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 -- 172.21.15.19:0/294740077 shutdown_connections 2024-09-19T04:07:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.396+0000 7f1650616740 1 -- 172.21.15.19:0/294740077 wait complete. 2024-09-19T04:07:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 1 -- start start 2024-09-19T04:07:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 1 Processor -- start 2024-09-19T04:07:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd09d0 0x564ffabf3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd12f0 0x564ffabf44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ffabeddb0 con 0x564ffabd09d0 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564ffabed6e0 con 0x564ffabdbbb0 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ffabee480 con 0x564ffabd12f0 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f1650616740 10 client.0 _ll_register_callbacks cb 0x7f163800cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f16477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd09d0 0x564ffabf3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f16477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd09d0 0x564ffabf3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57394/0 (socket says 172.21.15.19:57394) 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.397+0000 7f16477fe640 1 -- 172.21.15.19:0/2292110868 learned_addr learned my addr 172.21.15.19:0/2292110868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f163ffff640 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd12f0 0x564ffabf44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f1647fff640 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f16477fe640 1 -- 172.21.15.19:0/2292110868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd12f0 msgr2=0x564ffabf44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f16477fe640 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd12f0 0x564ffabf44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f16477fe640 1 -- 172.21.15.19:0/2292110868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 msgr2=0x564ffabef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f16477fe640 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f16477fe640 1 -- 172.21.15.19:0/2292110868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ffabefc10 con 0x564ffabd09d0 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f16477fe640 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd09d0 0x564ffabf3f90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f163809c820 tx=0x7f163809a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bac58639145b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f1626ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f1626ffd640 1 -- 172.21.15.19:0/2292110868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638018d60 con 0x564ffabd09d0 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f1626ffd640 1 -- 172.21.15.19:0/2292110868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f163800fb30 con 0x564ffabd09d0 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f1650616740 1 -- 172.21.15.19:0/2292110868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f163809a0d0 con 0x564ffabd09d0 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f1626ffd640 1 -- 172.21.15.19:0/2292110868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16380a0030 con 0x564ffabd09d0 2024-09-19T04:07:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.398+0000 7f1650616740 10 client.15381 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1650616740 1 -- 172.21.15.19:0/2292110868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ffabf35b0 con 0x564ffabd09d0 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1626ffd640 1 -- 172.21.15.19:0/2292110868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163809a0d0 con 0x564ffabd09d0 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1626ffd640 1 -- 172.21.15.19:0/2292110868 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f163806c070 con 0x564ffabd09d0 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1650616740 10 client.15381 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1650616740 10 client.15381 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1650616740 20 client.15381 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1650616740 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x564ffac5b0a0 0x564ffac59db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1650616740 4 client.15381 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f1650616740 1 -- 172.21.15.19:0/2292110868 --> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] -- command(tid 0: {"prefix": "status"}) -- 0x564ffab3eb10 con 0x564ffac5b0a0 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.399+0000 7f163ffff640 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x564ffac5b0a0 0x564ffac59db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f163ffff640 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x564ffac5b0a0 0x564ffac59db0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df3cb1f5169f86f1 server_cookie=527d270c632a0a53 in_seq=0 out_seq=0 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1626ffd640 10 client.15381 ms_handle_connect on v2:172.21.15.111:6836/1580295791 2024-09-19T04:07:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1626ffd640 1 -- 172.21.15.19:0/2292110868 <== mds.3 v2:172.21.15.111:6836/1580295791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x564ffab3eb10 con 0x564ffac5b0a0 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1626ffd640 10 client.15381 handle_command_reply: tid=0 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 client.15381 shutdown 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 -- 172.21.15.19:0/2292110868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd09d0 msgr2=0x564ffabf3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd09d0 0x564ffabf3f90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f163809c820 tx=0x7f163809a280 comp rx=0 tx=0).stop 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 -- 172.21.15.19:0/2292110868 shutdown_connections 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.111:6836/1580295791,v1:172.21.15.111:6837/1580295791] conn(0x564ffac5b0a0 0x564ffac59db0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ffabdbbb0 0x564ffabef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ffabd12f0 0x564ffabf44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 --2- 172.21.15.19:0/2292110868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ffabd09d0 0x564ffabf3f90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.400+0000 7f1650616740 1 -- 172.21.15.19:0/2292110868 >> 172.21.15.19:0/2292110868 conn(0x564ffabccf60 msgr2=0x564ffabd2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.401+0000 7f1626ffd640 0 client.15381 ms_handle_reset on v2:172.21.15.111:6836/1580295791 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.401+0000 7f1650616740 1 -- 172.21.15.19:0/2292110868 shutdown_connections 2024-09-19T04:07:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.401+0000 7f1650616740 1 -- 172.21.15.19:0/2292110868 wait complete. 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.401+0000 7f1650616740 20 client.15381 trim_cache size 0 max 16384 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24452, 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 118.89561667, 2024-09-19T04:07:20.407 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 29, 2024-09-19T04:07:20.408 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:07:20.408 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 93, 2024-09-19T04:07:20.408 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 181.62981508199999 2024-09-19T04:07:20.408 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:07:20.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:19 smithi111 ceph-mon[35365]: pgmap v372: 129 pgs: 129 active+clean; 664 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-19T04:07:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 -- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16340a3350 msgr2=0x7f16340a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16340a3350 0x7f16340a5810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1640005e30 tx=0x7f1640009880 comp rx=0 tx=0).stop 2024-09-19T04:07:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 -- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e20 msgr2=0x7f16480fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e20 0x7f16480fcc80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f163800c920 tx=0x7f163800cde0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 -- 172.21.15.19:0/3353713902 shutdown_connections 2024-09-19T04:07:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f16340a3350 0x7f16340a5810 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1648102ef0 0x7f16480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1648102570 0x7f16480fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 --2- 172.21.15.19:0/3353713902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e20 0x7f16480fcc80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:20.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 -- 172.21.15.19:0/3353713902 >> 172.21.15.19:0/3353713902 conn(0x7f16480fc7d0 msgr2=0x7f164806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:20.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 -- 172.21.15.19:0/3353713902 shutdown_connections 2024-09-19T04:07:20.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:20.402+0000 7f164e23d640 1 -- 172.21.15.19:0/3353713902 wait complete. 2024-09-19T04:07:20.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:19 smithi196 ceph-mon[35731]: pgmap v372: 129 pgs: 129 active+clean; 664 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-19T04:07:21.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T04:07:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:21.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 -- 172.21.15.19:0/1154959212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c103660 msgr2=0x7f2b7c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 --2- 172.21.15.19:0/1154959212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c103660 0x7f2b7c10bb50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009900 tx=0x7f2b7002a480 comp rx=0 tx=0).stop 2024-09-19T04:07:21.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 -- 172.21.15.19:0/1154959212 shutdown_connections 2024-09-19T04:07:21.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 --2- 172.21.15.19:0/1154959212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7c10c170 0x7f2b7c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 --2- 172.21.15.19:0/1154959212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c103660 0x7f2b7c10bb50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 --2- 172.21.15.19:0/1154959212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7c102d40 0x7f2b7c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 -- 172.21.15.19:0/1154959212 >> 172.21.15.19:0/1154959212 conn(0x7f2b7c0fc7a0 msgr2=0x7f2b7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:21.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 -- 172.21.15.19:0/1154959212 shutdown_connections 2024-09-19T04:07:21.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 -- 172.21.15.19:0/1154959212 wait complete. 2024-09-19T04:07:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.891+0000 7f2b82a78640 1 Processor -- start 2024-09-19T04:07:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b82a78640 1 -- start start 2024-09-19T04:07:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b82a78640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7c102d40 0x7f2b7c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b82a78640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7c103660 0x7f2b7c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b82a78640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c10c170 0x7f2b7c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b82a78640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c111f70 con 0x7f2b7c10c170 2024-09-19T04:07:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b82a78640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c111df0 con 0x7f2b7c102d40 2024-09-19T04:07:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b82a78640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1120f0 con 0x7f2b7c103660 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b81015640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c10c170 0x7f2b7c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b81015640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c10c170 0x7f2b7c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52882/0 (socket says 172.21.15.19:52882) 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b81015640 1 -- 172.21.15.19:0/3709466781 learned_addr learned my addr 172.21.15.19:0/3709466781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7c103660 0x7f2b7c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b80814640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7c102d40 0x7f2b7c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b81015640 1 -- 172.21.15.19:0/3709466781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7c103660 msgr2=0x7f2b7c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b81015640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7c103660 0x7f2b7c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b81015640 1 -- 172.21.15.19:0/3709466781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7c102d40 msgr2=0x7f2b7c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b81015640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7c102d40 0x7f2b7c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b81015640 1 -- 172.21.15.19:0/3709466781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c19d400 con 0x7f2b7c10c170 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7c103660 0x7f2b7c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.892+0000 7f2b80814640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7c102d40 0x7f2b7c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.893+0000 7f2b81015640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c10c170 0x7f2b7c19cd20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400eaa0 tx=0x7f2b7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.893+0000 7f2b79ffb640 1 -- 172.21.15.19:0/3709466781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400cdb0 con 0x7f2b7c10c170 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.893+0000 7f2b79ffb640 1 -- 172.21.15.19:0/3709466781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b74005020 con 0x7f2b7c10c170 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.893+0000 7f2b82a78640 1 -- 172.21.15.19:0/3709466781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c19d6f0 con 0x7f2b7c10c170 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.893+0000 7f2b79ffb640 1 -- 172.21.15.19:0/3709466781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740116b0 con 0x7f2b7c10c170 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.893+0000 7f2b82a78640 1 -- 172.21.15.19:0/3709466781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c1a4fd0 con 0x7f2b7c10c170 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.895+0000 7f2b79ffb640 1 -- 172.21.15.19:0/3709466781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f2b74005560 con 0x7f2b7c10c170 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b84e51740 1 Processor -- start 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b84e51740 1 -- start start 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b79ffb640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2b640a32d0 0x7f2b640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82fe8d0 0x5572b82fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82ff1f0 0x5572b830da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b83103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b84e51740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5572b8265090 con 0x5572b82fe8d0 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b84e51740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5572b820ca70 con 0x5572b82ff1f0 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b84e51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572b8261050 con 0x5572b830dfc0 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b79ffb640 1 -- 172.21.15.19:0/3709466781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f2b740d0100 con 0x7f2b7c10c170 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2b640a32d0 0x7f2b640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b80814640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82fe8d0 0x5572b82fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b80814640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82fe8d0 0x5572b82fecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52888/0 (socket says 172.21.15.19:52888) 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b80814640 1 -- 172.21.15.19:0/2038845905 learned_addr learned my addr 172.21.15.19:0/2038845905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82ff1f0 0x5572b830da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b81015640 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b83103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b80814640 1 -- 172.21.15.19:0/2038845905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 msgr2=0x5572b83103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b80814640 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b83103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b80814640 1 -- 172.21.15.19:0/2038845905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82ff1f0 msgr2=0x5572b830da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b80814640 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82ff1f0 0x5572b830da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b80814640 1 -- 172.21.15.19:0/2038845905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b8219130 con 0x5572b82fe8d0 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82ff1f0 0x5572b830da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.896+0000 7f2b81015640 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b83103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.897+0000 7f2b80814640 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82fe8d0 0x5572b82fecb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00bd40 tx=0x7f2b6c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682803109524075a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.897+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2b640a32d0 0x7f2b640a5790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0feda0 tx=0x7f2b70031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.897+0000 7f2b62ffd640 1 -- 172.21.15.19:0/2038845905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c015070 con 0x5572b82fe8d0 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.897+0000 7f2b62ffd640 1 -- 172.21.15.19:0/2038845905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c00be30 con 0x5572b82fe8d0 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.897+0000 7f2b62ffd640 1 -- 172.21.15.19:0/2038845905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c0107f0 con 0x5572b82fe8d0 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 -- 172.21.15.19:0/2038845905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82fe8d0 msgr2=0x5572b82fecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82fe8d0 0x5572b82fecb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00bd40 tx=0x7f2b6c00f730 comp rx=0 tx=0).stop 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 -- 172.21.15.19:0/2038845905 shutdown_connections 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b83103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82ff1f0 0x5572b830da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 --2- 172.21.15.19:0/2038845905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82fe8d0 0x5572b82fecb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 -- 172.21.15.19:0/2038845905 >> 172.21.15.19:0/2038845905 conn(0x5572b82faf60 msgr2=0x5572b82fb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 -- 172.21.15.19:0/2038845905 shutdown_connections 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 -- 172.21.15.19:0/2038845905 wait complete. 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 -- start start 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.898+0000 7f2b84e51740 1 Processor -- start 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 0x5572b830d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b830b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b84e51740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5572b8310bf0 con 0x5572b82ff1f0 2024-09-19T04:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b84e51740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5572b821ff20 con 0x5572b82fe8d0 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b84e51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572b8310d70 con 0x5572b830dfc0 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b84e51740 10 client.0 _ll_register_callbacks cb 0x5572b81e9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52902/0 (socket says 172.21.15.19:52902) 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1426598843 learned_addr learned my addr 172.21.15.19:0/1426598843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b80814640 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 0x5572b830d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b81015640 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b830b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 msgr2=0x5572b830b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b830b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1426598843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 msgr2=0x5572b830d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 0x5572b830d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1426598843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b7fd62b0 con 0x5572b82ff1f0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b81015640 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b830b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.899+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830ab60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b70033000 tx=0x7f2b70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bcae2d35adb4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1426598843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70035070 con 0x5572b82ff1f0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1426598843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f2b70004380 con 0x5572b82ff1f0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 1 -- 172.21.15.19:0/1426598843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b6c00be30 con 0x5572b82ff1f0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1426598843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b70057050 con 0x5572b82ff1f0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 10 client.15393 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 1 -- 172.21.15.19:0/1426598843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572b831c470 con 0x5572b82ff1f0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1426598843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c00be30 con 0x5572b82ff1f0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1426598843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2b70047040 con 0x5572b82ff1f0 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 10 client.15393 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 10 client.15393 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 20 client.15393 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x5572b831eb50 0x5572b831ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 4 client.15393 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.900+0000 7f2b84e51740 1 -- 172.21.15.19:0/1426598843 --> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b8272600 con 0x5572b831eb50 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.901+0000 7f2b80814640 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x5572b831eb50 0x5572b831ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.901+0000 7f2b5bfff640 10 client.15393 ms_handle_connect on v2:172.21.15.196:6836/1041250110 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.901+0000 7f2b80814640 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x5572b831eb50 0x5572b831ffc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f20e967f6f56a4c server_cookie=4bfd0f4075f5beb0 in_seq=0 out_seq=0 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.902+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1426598843 <== mds.2 v2:172.21.15.196:6836/1041250110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572b8272600 con 0x5572b831eb50 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.902+0000 7f2b5bfff640 10 client.15393 handle_command_reply: tid=0 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.902+0000 7f2b84e51740 1 client.15393 shutdown 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.902+0000 7f2b84e51740 1 -- 172.21.15.19:0/1426598843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 msgr2=0x5572b830ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.902+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830ab60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b70033000 tx=0x7f2b70009260 comp rx=0 tx=0).stop 2024-09-19T04:07:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.902+0000 7f2b84e51740 1 -- 172.21.15.19:0/1426598843 shutdown_connections 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x5572b831eb50 0x5572b831ffc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b830b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830ab60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1426598843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 0x5572b830d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b5bfff640 0 client.15393 ms_handle_reset on v2:172.21.15.196:6836/1041250110 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b84e51740 1 -- 172.21.15.19:0/1426598843 >> 172.21.15.19:0/1426598843 conn(0x5572b82faf60 msgr2=0x5572b8300cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b84e51740 1 -- 172.21.15.19:0/1426598843 shutdown_connections 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b84e51740 1 -- 172.21.15.19:0/1426598843 wait complete. 2024-09-19T04:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.903+0000 7f2b84e51740 20 client.15393 trim_cache size 0 max 16384 2024-09-19T04:07:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b84e51740 1 Processor -- start 2024-09-19T04:07:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b84e51740 1 -- start start 2024-09-19T04:07:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 0x5572b82ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b8309430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b84e51740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5572b831e300 con 0x5572b82ff1f0 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b84e51740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5572b831e9d0 con 0x5572b82fe8d0 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b84e51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572b831f0a0 con 0x5572b830dfc0 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52914/0 (socket says 172.21.15.19:52914) 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1877855685 learned_addr learned my addr 172.21.15.19:0/1877855685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b80814640 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 0x5572b82ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b81015640 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b8309430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1877855685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 msgr2=0x5572b8309430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b8309430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1877855685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 msgr2=0x5572b82ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 0x5572b82ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1877855685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b82fc0f0 con 0x5572b82ff1f0 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.921+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830a420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009230 tx=0x7f2b70009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2828d26bff7ac3b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b627fc640 1 -- 172.21.15.19:0/1877855685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70060070 con 0x5572b82ff1f0 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b627fc640 1 -- 172.21.15.19:0/1877855685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b70004380 con 0x5572b82ff1f0 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b627fc640 1 -- 172.21.15.19:0/1877855685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70004e00 con 0x5572b82ff1f0 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 -- 172.21.15.19:0/1877855685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 msgr2=0x5572b830a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830a420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009230 tx=0x7f2b70009780 comp rx=0 tx=0).stop 2024-09-19T04:07:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 -- 172.21.15.19:0/1877855685 shutdown_connections 2024-09-19T04:07:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b830dfc0 0x5572b8309430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b830a420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1877855685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b82fe8d0 0x5572b82ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 -- 172.21.15.19:0/1877855685 >> 172.21.15.19:0/1877855685 conn(0x5572b82faf60 msgr2=0x5572b82fb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 -- 172.21.15.19:0/1877855685 shutdown_connections 2024-09-19T04:07:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 -- 172.21.15.19:0/1877855685 wait complete. 2024-09-19T04:07:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.922+0000 7f2b84e51740 1 -- start start 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b84e51740 1 Processor -- start 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b82fe8d0 0x5572b83211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b8321700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b84e51740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b830dfc0 0x5572b8321c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b84e51740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5572b831fe40 con 0x5572b82ff1f0 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b84e51740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5572b831f770 con 0x5572b830dfc0 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b84e51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572b8320510 con 0x5572b82fe8d0 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b84e51740 10 client.0 _ll_register_callbacks cb 0x7f2b7c06b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b8321700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b8321700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52928/0 (socket says 172.21.15.19:52928) 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1212636992 learned_addr learned my addr 172.21.15.19:0/1212636992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b81015640 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b830dfc0 0x5572b8321c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.923+0000 7f2b80814640 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b82fe8d0 0x5572b83211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1212636992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b82fe8d0 msgr2=0x5572b83211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b82fe8d0 0x5572b83211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1212636992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b830dfc0 msgr2=0x5572b8321c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b830dfc0 0x5572b8321c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b7bfff640 1 -- 172.21.15.19:0/1212636992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b82fad90 con 0x5572b82ff1f0 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b7bfff640 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b8321700 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2b70003890 tx=0x7f2b700045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866e4bb9cca4258e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1212636992 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70060070 con 0x5572b82ff1f0 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1212636992 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f2b700046c0 con 0x5572b82ff1f0 2024-09-19T04:07:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b84e51740 1 -- 172.21.15.19:0/1212636992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b70004380 con 0x5572b82ff1f0 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1212636992 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b70035050 con 0x5572b82ff1f0 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b84e51740 10 client.15399 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b84e51740 1 -- 172.21.15.19:0/1212636992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572b83883d0 con 0x5572b82ff1f0 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1212636992 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b70004380 con 0x5572b82ff1f0 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.924+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1212636992 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2b70047040 con 0x5572b82ff1f0 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b84e51740 10 client.15399 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b84e51740 10 client.15399 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b84e51740 20 client.15399 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x5572b838cb40 0x5572b838b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b84e51740 4 client.15399 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b84e51740 1 -- 172.21.15.19:0/1212636992 --> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] -- command(tid 0: {"prefix": "status"}) -- 0x5572b826cb10 con 0x5572b838cb40 2024-09-19T04:07:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b80814640 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x5572b838cb40 0x5572b838b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b5bfff640 10 client.15399 ms_handle_connect on v2:172.21.15.196:6836/1041250110 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.925+0000 7f2b80814640 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x5572b838cb40 0x5572b838b850 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf4f32b973dcfb42 server_cookie=41b6f3371d58bce7 in_seq=0 out_seq=0 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b5bfff640 1 -- 172.21.15.19:0/1212636992 <== mds.2 v2:172.21.15.196:6836/1041250110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5572b826cb10 con 0x5572b838cb40 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b5bfff640 10 client.15399 handle_command_reply: tid=0 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 client.15399 shutdown 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 -- 172.21.15.19:0/1212636992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 msgr2=0x5572b8321700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b8321700 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2b70003890 tx=0x7f2b700045a0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 -- 172.21.15.19:0/1212636992 shutdown_connections 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.196:6836/1041250110,v1:172.21.15.196:6837/1041250110] conn(0x5572b838cb40 0x5572b838b850 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b5bfff640 0 client.15399 ms_handle_reset on v2:172.21.15.196:6836/1041250110 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5572b830dfc0 0x5572b8321c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572b82ff1f0 0x5572b8321700 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 --2- 172.21.15.19:0/1212636992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572b82fe8d0 0x5572b83211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 -- 172.21.15.19:0/1212636992 >> 172.21.15.19:0/1212636992 conn(0x5572b82faf60 msgr2=0x5572b82fb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 -- 172.21.15.19:0/1212636992 shutdown_connections 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 1 -- 172.21.15.19:0/1212636992 wait complete. 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.926+0000 7f2b84e51740 20 client.15399 trim_cache size 0 max 16384 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24454, 2024-09-19T04:07:21.934 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:07:21.935 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:07:21.935 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:07:21.935 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 121.426723638, 2024-09-19T04:07:21.935 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 29, 2024-09-19T04:07:21.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:07:21.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 93, 2024-09-19T04:07:21.935 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 185.96653065699999 2024-09-19T04:07:21.935 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:07:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 -- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2b640a32d0 msgr2=0x7f2b640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2b640a32d0 0x7f2b640a5790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0feda0 tx=0x7f2b70031510 comp rx=0 tx=0).stop 2024-09-19T04:07:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 -- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c10c170 msgr2=0x7f2b7c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c10c170 0x7f2b7c19cd20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400eaa0 tx=0x7f2b7400ef60 comp rx=0 tx=0).stop 2024-09-19T04:07:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 -- 172.21.15.19:0/3709466781 shutdown_connections 2024-09-19T04:07:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f2b640a32d0 0x7f2b640a5790 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c10c170 0x7f2b7c19cd20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7c103660 0x7f2b7c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 --2- 172.21.15.19:0/3709466781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7c102d40 0x7f2b7c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 -- 172.21.15.19:0/3709466781 >> 172.21.15.19:0/3709466781 conn(0x7f2b7c0fc7a0 msgr2=0x7f2b7c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 -- 172.21.15.19:0/3709466781 shutdown_connections 2024-09-19T04:07:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:21.927+0000 7f2b82a78640 1 -- 172.21.15.19:0/3709466781 wait complete. 2024-09-19T04:07:22.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:21 smithi019 ceph-mon[30938]: pgmap v373: 129 pgs: 129 active+clean; 664 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T04:07:22.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:21 smithi111 ceph-mon[35365]: pgmap v373: 129 pgs: 129 active+clean; 664 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T04:07:22.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:21 smithi196 ceph-mon[35731]: pgmap v373: 129 pgs: 129 active+clean; 664 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T04:07:22.666 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T04:07:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 -- 172.21.15.19:0/2356136381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34102790 msgr2=0x7fbc341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2356136381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34102790 0x7fbc341067e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009900 tx=0x7fbc2002a410 comp rx=0 tx=0).stop 2024-09-19T04:07:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 -- 172.21.15.19:0/2356136381 shutdown_connections 2024-09-19T04:07:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2356136381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc34106f10 0x7fbc3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2356136381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34102790 0x7fbc341067e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2356136381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc34101de0 0x7fbc341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 -- 172.21.15.19:0/2356136381 >> 172.21.15.19:0/2356136381 conn(0x7fbc340fd8c0 msgr2=0x7fbc340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 -- 172.21.15.19:0/2356136381 shutdown_connections 2024-09-19T04:07:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.397+0000 7fbc3af11640 1 -- 172.21.15.19:0/2356136381 wait complete. 2024-09-19T04:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc3af11640 1 Processor -- start 2024-09-19T04:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc3af11640 1 -- start start 2024-09-19T04:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc3af11640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34101de0 0x7fbc341984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc3af11640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc34102790 0x7fbc341989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc3af11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc34106f10 0x7fbc3419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc3af11640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34111f70 con 0x7fbc34101de0 2024-09-19T04:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc3af11640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34111df0 con 0x7fbc34102790 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc3af11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341120f0 con 0x7fbc34106f10 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34101de0 0x7fbc341984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34101de0 0x7fbc341984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52950/0 (socket says 172.21.15.19:52950) 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 -- 172.21.15.19:0/2886226238 learned_addr learned my addr 172.21.15.19:0/2886226238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc34102790 0x7fbc341989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc394ae640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc34106f10 0x7fbc3419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 -- 172.21.15.19:0/2886226238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc34106f10 msgr2=0x7fbc3419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc34106f10 0x7fbc3419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 -- 172.21.15.19:0/2886226238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc34102790 msgr2=0x7fbc341989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc34102790 0x7fbc341989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 -- 172.21.15.19:0/2886226238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3419d460 con 0x7fbc34101de0 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc34102790 0x7fbc341989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc394ae640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc34106f10 0x7fbc3419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.398+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34101de0 0x7fbc341984b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e9f0 tx=0x7fbc2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.399+0000 7fbc31ffb640 1 -- 172.21.15.19:0/2886226238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800cd80 con 0x7fbc34101de0 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.399+0000 7fbc31ffb640 1 -- 172.21.15.19:0/2886226238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc28005500 con 0x7fbc34101de0 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.399+0000 7fbc3af11640 1 -- 172.21.15.19:0/2886226238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3419d750 con 0x7fbc34101de0 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.399+0000 7fbc31ffb640 1 -- 172.21.15.19:0/2886226238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28010690 con 0x7fbc34101de0 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.399+0000 7fbc3af11640 1 -- 172.21.15.19:0/2886226238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc341a5030 con 0x7fbc34101de0 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.401+0000 7fbc31ffb640 1 -- 172.21.15.19:0/2886226238 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fbc280046c0 con 0x7fbc34101de0 2024-09-19T04:07:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.401+0000 7fbc3d2ea740 1 Processor -- start 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.401+0000 7fbc3d2ea740 1 -- start start 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.401+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4669d0 0x56042e466db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.401+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4672f0 0x56042e471670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.401+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e473fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56042e3cd090 con 0x56042e4672f0 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56042e374a70 con 0x56042e471bb0 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56042e3c9050 con 0x56042e4669d0 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4672f0 0x56042e471670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc31ffb640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbc240a3350 0x7fbc240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4672f0 0x56042e471670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52956/0 (socket says 172.21.15.19:52956) 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 -- 172.21.15.19:0/2227249910 learned_addr learned my addr 172.21.15.19:0/2227249910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:23.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e473fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc394ae640 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4669d0 0x56042e466db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc31ffb640 1 -- 172.21.15.19:0/2886226238 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fbc280d0310 con 0x7fbc34101de0 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc394ae640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbc240a3350 0x7fbc240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 -- 172.21.15.19:0/2227249910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4669d0 msgr2=0x56042e466db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4669d0 0x56042e466db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 -- 172.21.15.19:0/2227249910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 msgr2=0x56042e473fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e473fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 -- 172.21.15.19:0/2227249910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56042e381130 con 0x56042e4672f0 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e473fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4672f0 0x56042e471670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbc280cd890 tx=0x7fbc28005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9ae521f40a128d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc394ae640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbc240a3350 0x7fbc240a5810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005c40 tx=0x7fbc2c009200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.402+0000 7fbc1effd640 1 -- 172.21.15.19:0/2227249910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2809bd90 con 0x56042e4672f0 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc1effd640 1 -- 172.21.15.19:0/2227249910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2809cb30 con 0x56042e4672f0 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc1effd640 1 -- 172.21.15.19:0/2227249910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280cdb70 con 0x56042e4672f0 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2227249910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4672f0 msgr2=0x56042e471670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4672f0 0x56042e471670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbc280cd890 tx=0x7fbc28005730 comp rx=0 tx=0).stop 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2227249910 shutdown_connections 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e473fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4672f0 0x56042e471670 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2227249910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4669d0 0x56042e466db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2227249910 >> 172.21.15.19:0/2227249910 conn(0x56042e462f60 msgr2=0x56042e463340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2227249910 shutdown_connections 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.403+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2227249910 wait complete. 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 1 -- start start 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 1 Processor -- start 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e4713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e4672f0 0x56042e46e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e471bb0 0x56042e46ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56042e474720 con 0x56042e4669d0 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56042e387f20 con 0x56042e4672f0 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56042e4748a0 con 0x56042e471bb0 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc3d2ea740 10 client.0 _ll_register_callbacks cb 0x56042e351030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.404+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e4713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e4713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52958/0 (socket says 172.21.15.19:52958) 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc38cad640 1 -- 172.21.15.19:0/2740047937 learned_addr learned my addr 172.21.15.19:0/2740047937 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e4672f0 0x56042e46e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc38cad640 1 -- 172.21.15.19:0/2740047937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e471bb0 msgr2=0x56042e46ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e471bb0 0x56042e46ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc38cad640 1 -- 172.21.15.19:0/2740047937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e4672f0 msgr2=0x56042e46e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e4672f0 0x56042e46e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc38cad640 1 -- 172.21.15.19:0/2740047937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56042e13e2b0 con 0x56042e4669d0 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e4672f0 0x56042e46e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e4713d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbc280cd9a0 tx=0x7fbc2800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207f7d7aa212e87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc13fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc13fff640 1 -- 172.21.15.19:0/2740047937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2809bd90 con 0x56042e4669d0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc13fff640 1 -- 172.21.15.19:0/2740047937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fbc280a4e20 con 0x56042e4669d0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2740047937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2809cb30 con 0x56042e4669d0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc13fff640 1 -- 172.21.15.19:0/2740047937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc2809c430 con 0x56042e4669d0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc3d2ea740 10 client.15411 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2740047937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56042e47ffd0 con 0x56042e4669d0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.405+0000 7fbc13fff640 1 -- 172.21.15.19:0/2740047937 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2809cb30 con 0x56042e4669d0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc13fff640 1 -- 172.21.15.19:0/2740047937 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbc2809b460 con 0x56042e4669d0 2024-09-19T04:07:23.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc3d2ea740 10 client.15411 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc3d2ea740 10 client.15411 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc3d2ea740 20 client.15411 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x56042e4821e0 0x56042e483650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc3d2ea740 4 client.15411 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2740047937 --> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56042e3da600 con 0x56042e4821e0 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x56042e4821e0 0x56042e483650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x56042e4821e0 0x56042e483650 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c7dda0822f5cc06 server_cookie=a3c389b4e5ffe0d8 in_seq=0 out_seq=0 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.406+0000 7fbc13fff640 10 client.15411 ms_handle_connect on v2:172.21.15.196:6838/1173261295 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc13fff640 1 -- 172.21.15.19:0/2740047937 <== mds.1 v2:172.21.15.196:6838/1173261295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56042e3da600 con 0x56042e4821e0 2024-09-19T04:07:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc13fff640 10 client.15411 handle_command_reply: tid=0 2024-09-19T04:07:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 client.15411 shutdown 2024-09-19T04:07:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2740047937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 msgr2=0x56042e4713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e4713d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbc280cd9a0 tx=0x7fbc2800cf20 comp rx=0 tx=0).stop 2024-09-19T04:07:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2740047937 shutdown_connections 2024-09-19T04:07:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x56042e4821e0 0x56042e483650 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e471bb0 0x56042e46ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e4672f0 0x56042e46e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc13fff640 0 client.15411 ms_handle_reset on v2:172.21.15.196:6838/1173261295 2024-09-19T04:07:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2740047937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e4713d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.407+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2740047937 >> 172.21.15.19:0/2740047937 conn(0x56042e462f60 msgr2=0x56042e468c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.408+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2740047937 shutdown_connections 2024-09-19T04:07:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.408+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2740047937 wait complete. 2024-09-19T04:07:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.408+0000 7fbc3d2ea740 20 client.15411 trim_cache size 0 max 16384 2024-09-19T04:07:23.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.426+0000 7fbc3d2ea740 1 Processor -- start 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.426+0000 7fbc3d2ea740 1 -- start start 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.426+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e483650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.426+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e469c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.426+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e46b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.426+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56042e481990 con 0x56042e4669d0 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.426+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56042e482060 con 0x56042e471bb0 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.426+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56042e482730 con 0x56042e4672f0 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e483650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e483650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52960/0 (socket says 172.21.15.19:52960) 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 -- 172.21.15.19:0/2099130586 learned_addr learned my addr 172.21.15.19:0/2099130586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e469c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 -- 172.21.15.19:0/2099130586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 msgr2=0x56042e469c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e469c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 -- 172.21.15.19:0/2099130586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 msgr2=0x56042e46b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e46b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 -- 172.21.15.19:0/2099130586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56042e471740 con 0x56042e4669d0 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc33fff640 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e469c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc38cad640 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e483650 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbc2809c130 tx=0x7fbc28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec96aced7e11da8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/2099130586 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2809bd90 con 0x56042e4669d0 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/2099130586 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2809cb30 con 0x56042e4669d0 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.427+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/2099130586 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2809b880 con 0x56042e4669d0 2024-09-19T04:07:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2099130586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 msgr2=0x56042e483650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e483650 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbc2809c130 tx=0x7fbc28004a40 comp rx=0 tx=0).stop 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2099130586 shutdown_connections 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e46b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e469c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/2099130586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e483650 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2099130586 >> 172.21.15.19:0/2099130586 conn(0x56042e462f60 msgr2=0x56042e469560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2099130586 shutdown_connections 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/2099130586 wait complete. 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 -- start start 2024-09-19T04:07:23.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.428+0000 7fbc3d2ea740 1 Processor -- start 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e48bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e46b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc3d2ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e46b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56042e4834d0 con 0x56042e4669d0 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56042e482e00 con 0x56042e471bb0 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc3d2ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56042e483ba0 con 0x56042e4672f0 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc3d2ea740 10 client.0 _ll_register_callbacks cb 0x7fbc28005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e48bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc38cad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e48bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52966/0 (socket says 172.21.15.19:52966) 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc38cad640 1 -- 172.21.15.19:0/542815711 learned_addr learned my addr 172.21.15.19:0/542815711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc394ae640 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e46b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc33fff640 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e46b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc38cad640 1 -- 172.21.15.19:0/542815711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 msgr2=0x56042e46b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc38cad640 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e46b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc38cad640 1 -- 172.21.15.19:0/542815711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 msgr2=0x56042e46b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc38cad640 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e46b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc38cad640 1 -- 172.21.15.19:0/542815711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56042e46bc90 con 0x56042e4669d0 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.429+0000 7fbc394ae640 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e46b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc38cad640 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e48bd20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbc280049d0 tx=0x7fbc28004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4869ff2e2c399e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc13fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc13fff640 1 -- 172.21.15.19:0/542815711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2809bd90 con 0x56042e4669d0 2024-09-19T04:07:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc13fff640 1 -- 172.21.15.19:0/542815711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fbc2809b880 con 0x56042e4669d0 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/542815711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2809cb30 con 0x56042e4669d0 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc13fff640 1 -- 172.21.15.19:0/542815711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc2809ba80 con 0x56042e4669d0 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 10 client.15417 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/542815711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56042e46bc90 con 0x56042e4669d0 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc13fff640 1 -- 172.21.15.19:0/542815711 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2809cb30 con 0x56042e4669d0 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc13fff640 1 -- 172.21.15.19:0/542815711 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbc28004f40 con 0x56042e4669d0 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 10 client.15417 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 10 client.15417 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 20 client.15417 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x56042e4f0a80 0x56042e4ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 4 client.15417 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.430+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/542815711 --> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] -- command(tid 0: {"prefix": "status"}) -- 0x56042e3d4b10 con 0x56042e4f0a80 2024-09-19T04:07:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.431+0000 7fbc33fff640 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x56042e4f0a80 0x56042e4ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.431+0000 7fbc13fff640 10 client.15417 ms_handle_connect on v2:172.21.15.196:6838/1173261295 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.431+0000 7fbc33fff640 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x56042e4f0a80 0x56042e4ef790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca00d6efd23e0f6c server_cookie=264ecffce73740ba in_seq=0 out_seq=0 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.431+0000 7fbc13fff640 1 -- 172.21.15.19:0/542815711 <== mds.1 v2:172.21.15.196:6838/1173261295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56042e3d4b10 con 0x56042e4f0a80 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.431+0000 7fbc13fff640 10 client.15417 handle_command_reply: tid=0 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.431+0000 7fbc3d2ea740 1 client.15417 shutdown 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.431+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/542815711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 msgr2=0x56042e48bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.431+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e48bd20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbc280049d0 tx=0x7fbc28004ba0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/542815711 shutdown_connections 2024-09-19T04:07:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.196:6838/1173261295,v1:172.21.15.196:6839/1173261295] conn(0x56042e4f0a80 0x56042e4ef790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042e471bb0 0x56042e46b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56042e4672f0 0x56042e46b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 1 --2- 172.21.15.19:0/542815711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56042e4669d0 0x56042e48bd20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc13fff640 0 client.15417 ms_handle_reset on v2:172.21.15.196:6838/1173261295 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/542815711 >> 172.21.15.19:0/542815711 conn(0x56042e462f60 msgr2=0x56042e468c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/542815711 shutdown_connections 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 1 -- 172.21.15.19:0/542815711 wait complete. 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3d2ea740 20 client.15417 trim_cache size 0 max 16384 2024-09-19T04:07:23.445 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24469, 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 123.94099810100001, 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 29, 2024-09-19T04:07:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:07:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 93, 2024-09-19T04:07:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 182.09918829200001 2024-09-19T04:07:23.447 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:07:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 -- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbc240a3350 msgr2=0x7fbc240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbc240a3350 0x7fbc240a5810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005c40 tx=0x7fbc2c009200 comp rx=0 tx=0).stop 2024-09-19T04:07:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 -- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34101de0 msgr2=0x7fbc341984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34101de0 0x7fbc341984b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e9f0 tx=0x7fbc2800edf0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 -- 172.21.15.19:0/2886226238 shutdown_connections 2024-09-19T04:07:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbc240a3350 0x7fbc240a5810 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc34106f10 0x7fbc3419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc34102790 0x7fbc341989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 --2- 172.21.15.19:0/2886226238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34101de0 0x7fbc341984b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:23.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.432+0000 7fbc3af11640 1 -- 172.21.15.19:0/2886226238 >> 172.21.15.19:0/2886226238 conn(0x7fbc340fd8c0 msgr2=0x7fbc340fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:23.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.433+0000 7fbc3af11640 1 -- 172.21.15.19:0/2886226238 shutdown_connections 2024-09-19T04:07:23.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:23.433+0000 7fbc3af11640 1 -- 172.21.15.19:0/2886226238 wait complete. 2024-09-19T04:07:24.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:23 smithi019 ceph-mon[30938]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-09-19T04:07:24.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T04:07:24.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:23 smithi196 ceph-mon[35731]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-09-19T04:07:24.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:23 smithi111 ceph-mon[35365]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-09-19T04:07:24.457 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 -- 172.21.15.19:0/233365279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200108290 msgr2=0x7f520010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 --2- 172.21.15.19:0/233365279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200108290 0x7f520010ec30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009cd0 tx=0x7f51f802a8e0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 -- 172.21.15.19:0/233365279 shutdown_connections 2024-09-19T04:07:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 --2- 172.21.15.19:0/233365279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200108290 0x7f520010ec30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 --2- 172.21.15.19:0/233365279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200103740 0x7f5200107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 --2- 172.21.15.19:0/233365279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200102d90 0x7f5200103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 -- 172.21.15.19:0/233365279 >> 172.21.15.19:0/233365279 conn(0x7f52000fc7d0 msgr2=0x7f52000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 -- 172.21.15.19:0/233365279 shutdown_connections 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 -- 172.21.15.19:0/233365279 wait complete. 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.925+0000 7f5206b9b640 1 Processor -- start 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5206b9b640 1 -- start start 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5206b9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200102d90 0x7f52001985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5206b9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103740 0x7f5200198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5206b9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200108290 0x7f520019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5206b9b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52001120d0 con 0x7f5200102d90 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5206b9b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5200111f50 con 0x7f5200108290 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5206b9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5200112250 con 0x7f5200103740 2024-09-19T04:07:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200102d90 0x7f52001985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200102d90 0x7f52001985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52984/0 (socket says 172.21.15.19:52984) 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 -- 172.21.15.19:0/2254647843 learned_addr learned my addr 172.21.15.19:0/2254647843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5205138640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200108290 0x7f520019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f51fffff640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103740 0x7f5200198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 -- 172.21.15.19:0/2254647843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103740 msgr2=0x7f5200198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103740 0x7f5200198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 -- 172.21.15.19:0/2254647843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200108290 msgr2=0x7f520019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200108290 0x7f520019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 -- 172.21.15.19:0/2254647843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520019d560 con 0x7f5200102d90 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.926+0000 7f5204937640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200102d90 0x7f52001985b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f51f000b4a0 tx=0x7f51f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:24.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.927+0000 7f51fdffb640 1 -- 172.21.15.19:0/2254647843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0014070 con 0x7f5200102d90 2024-09-19T04:07:24.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.927+0000 7f51fdffb640 1 -- 172.21.15.19:0/2254647843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f0010af0 con 0x7f5200102d90 2024-09-19T04:07:24.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.927+0000 7f51fdffb640 1 -- 172.21.15.19:0/2254647843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0017d10 con 0x7f5200102d90 2024-09-19T04:07:24.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.927+0000 7f5206b9b640 1 -- 172.21.15.19:0/2254647843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520019d850 con 0x7f5200102d90 2024-09-19T04:07:24.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.927+0000 7f5206b9b640 1 -- 172.21.15.19:0/2254647843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001a5130 con 0x7f5200102d90 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.929+0000 7f51fdffb640 1 -- 172.21.15.19:0/2254647843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f51f0017070 con 0x7f5200102d90 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5208f74740 1 Processor -- start 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5208f74740 1 -- start start 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5208f74740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea87cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f51fdffb640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51ec0a32d0 0x7f51ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5208f74740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea881f0 0x55779ea96a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5208f74740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 0x55779ea993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5208f74740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55779e9ee090 con 0x55779ea881f0 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5208f74740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55779e995a70 con 0x55779ea96fc0 2024-09-19T04:07:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5208f74740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55779e9ea050 con 0x55779ea878d0 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5204937640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51ec0a32d0 0x7f51ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f51fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea881f0 0x55779ea96a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f5205138640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea87cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f51fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea881f0 0x55779ea96a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52986/0 (socket says 172.21.15.19:52986) 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f51fdffb640 1 -- 172.21.15.19:0/2254647843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f51f00cf100 con 0x7f5200102d90 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.930+0000 7f51fffff640 1 -- 172.21.15.19:0/1443176577 learned_addr learned my addr 172.21.15.19:0/1443176577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f5205138640 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 0x55779ea993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51fffff640 1 -- 172.21.15.19:0/1443176577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 msgr2=0x55779ea87cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51fffff640 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea87cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51fffff640 1 -- 172.21.15.19:0/1443176577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 msgr2=0x55779ea993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51fffff640 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 0x55779ea993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51fffff640 1 -- 172.21.15.19:0/1443176577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55779e9a2130 con 0x55779ea881f0 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f5205138640 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea87cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f5204937640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51ec0a32d0 0x7f51ec0a5790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f51f0097d00 tx=0x7f51f0013500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51fffff640 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea881f0 0x55779ea96a80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f51e800bc40 tx=0x7f51e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831226b92d122e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51f6ffd640 1 -- 172.21.15.19:0/1443176577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8015070 con 0x55779ea881f0 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51f6ffd640 1 -- 172.21.15.19:0/1443176577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e8011920 con 0x55779ea881f0 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.931+0000 7f51f6ffd640 1 -- 172.21.15.19:0/1443176577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8010890 con 0x55779ea881f0 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 -- 172.21.15.19:0/1443176577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea881f0 msgr2=0x55779ea96a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea881f0 0x55779ea96a80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f51e800bc40 tx=0x7f51e800f730 comp rx=0 tx=0).stop 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 -- 172.21.15.19:0/1443176577 shutdown_connections 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 0x55779ea993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea881f0 0x55779ea96a80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 --2- 172.21.15.19:0/1443176577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea87cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 -- 172.21.15.19:0/1443176577 >> 172.21.15.19:0/1443176577 conn(0x55779ea83f60 msgr2=0x55779ea84340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 -- 172.21.15.19:0/1443176577 shutdown_connections 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 -- 172.21.15.19:0/1443176577 wait complete. 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.932+0000 7f5208f74740 1 -- start start 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5208f74740 1 Processor -- start 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5208f74740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5208f74740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 0x55779ea93bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5208f74740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779ea940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5208f74740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55779ea99bf0 con 0x55779ea96fc0 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5208f74740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55779e9a8f20 con 0x55779ea881f0 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5208f74740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55779ea99d70 con 0x55779ea878d0 2024-09-19T04:07:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5208f74740 10 client.0 _ll_register_callbacks cb 0x55779e972030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779ea940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779ea940f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52996/0 (socket says 172.21.15.19:52996) 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 -- 172.21.15.19:0/4111954978 learned_addr learned my addr 172.21.15.19:0/4111954978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f51fffff640 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 0x55779ea93bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 -- 172.21.15.19:0/4111954978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 msgr2=0x55779ea96860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea96860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 -- 172.21.15.19:0/4111954978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 msgr2=0x55779ea93bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 0x55779ea93bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.933+0000 7f5205138640 1 -- 172.21.15.19:0/4111954978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55779e75f2b0 con 0x55779ea96fc0 2024-09-19T04:07:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f5205138640 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779ea940f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009a60 tx=0x7f51f80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6e182e18369f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f51dffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f51dffff640 1 -- 172.21.15.19:0/4111954978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8051060 con 0x55779ea96fc0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f5208f74740 1 -- 172.21.15.19:0/4111954978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51e8011920 con 0x55779ea96fc0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f51dffff640 1 -- 172.21.15.19:0/4111954978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f51f8030a70 con 0x55779ea96fc0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f51dffff640 1 -- 172.21.15.19:0/4111954978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f51f8030c70 con 0x55779ea96fc0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f5208f74740 10 client.15429 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f5208f74740 1 -- 172.21.15.19:0/4111954978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55779eaa5550 con 0x55779ea96fc0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f51dffff640 1 -- 172.21.15.19:0/4111954978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e8011920 con 0x55779ea96fc0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f51dffff640 1 -- 172.21.15.19:0/4111954978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f51f8030070 con 0x55779ea96fc0 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f5208f74740 10 client.15429 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.934+0000 7f5208f74740 10 client.15429 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:07:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.935+0000 7f5208f74740 20 client.15429 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.935+0000 7f5208f74740 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55779eaa7b00 0x55779eaa8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.935+0000 7f5208f74740 4 client.15429 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.935+0000 7f5208f74740 1 -- 172.21.15.19:0/4111954978 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55779e9fb600 con 0x55779eaa7b00 2024-09-19T04:07:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.935+0000 7f51fffff640 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55779eaa7b00 0x55779eaa8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.935+0000 7f51fffff640 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55779eaa7b00 0x55779eaa8f70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad178a3514ba8a5 server_cookie=a87f68ece7885589 in_seq=0 out_seq=0 2024-09-19T04:07:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.935+0000 7f51dffff640 10 client.15429 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f51dffff640 1 -- 172.21.15.19:0/4111954978 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55779e9fb600 con 0x55779eaa7b00 2024-09-19T04:07:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f51dffff640 10 client.15429 handle_command_reply: tid=0 2024-09-19T04:07:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 client.15429 shutdown 2024-09-19T04:07:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 -- 172.21.15.19:0/4111954978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 msgr2=0x55779ea940f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779ea940f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009a60 tx=0x7f51f80095c0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 -- 172.21.15.19:0/4111954978 shutdown_connections 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779ea940f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 0x55779ea93bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55779eaa7b00 0x55779eaa8f70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 --2- 172.21.15.19:0/4111954978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779ea96860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 -- 172.21.15.19:0/4111954978 >> 172.21.15.19:0/4111954978 conn(0x55779ea83f60 msgr2=0x55779ea89c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f51dffff640 0 client.15429 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 -- 172.21.15.19:0/4111954978 shutdown_connections 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.936+0000 7f5208f74740 1 -- 172.21.15.19:0/4111954978 wait complete. 2024-09-19T04:07:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.937+0000 7f5208f74740 20 client.15429 trim_cache size 0 max 16384 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.954+0000 7f5208f74740 1 Processor -- start 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.954+0000 7f5208f74740 1 -- start start 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5208f74740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea878d0 0x55779eaa7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5208f74740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea881f0 0x55779eab1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5208f74740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 0x55779eab1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5208f74740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55779eaa72b0 con 0x55779ea878d0 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5208f74740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55779eaa7980 con 0x55779ea96fc0 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5208f74740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55779eaa8050 con 0x55779ea881f0 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea878d0 0x55779eaa7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea878d0 0x55779eaa7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53012/0 (socket says 172.21.15.19:53012) 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 -- 172.21.15.19:0/3680622717 learned_addr learned my addr 172.21.15.19:0/3680622717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f51fffff640 1 --2- 172.21.15.19:0/3680622717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea881f0 0x55779eab1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 -- 172.21.15.19:0/3680622717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea881f0 msgr2=0x55779eab1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 --2- 172.21.15.19:0/3680622717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea881f0 0x55779eab1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 -- 172.21.15.19:0/3680622717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 msgr2=0x55779eab1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 --2- 172.21.15.19:0/3680622717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 0x55779eab1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 -- 172.21.15.19:0/3680622717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55779ea88020 con 0x55779ea878d0 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.955+0000 7f5205138640 1 --2- 172.21.15.19:0/3680622717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea878d0 0x55779eaa7430 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f51f8005f50 tx=0x7f51f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b754d83ae92f7b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f51f67fc640 1 -- 172.21.15.19:0/3680622717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8051040 con 0x55779ea878d0 2024-09-19T04:07:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f51f67fc640 1 -- 172.21.15.19:0/3680622717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f8036070 con 0x55779ea878d0 2024-09-19T04:07:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f51f67fc640 1 -- 172.21.15.19:0/3680622717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f804acb0 con 0x55779ea878d0 2024-09-19T04:07:24.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 -- 172.21.15.19:0/3680622717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea878d0 msgr2=0x55779eaa7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 --2- 172.21.15.19:0/3680622717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea878d0 0x55779eaa7430 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f51f8005f50 tx=0x7f51f8004a40 comp rx=0 tx=0).stop 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 -- 172.21.15.19:0/3680622717 shutdown_connections 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 --2- 172.21.15.19:0/3680622717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea96fc0 0x55779eab1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 --2- 172.21.15.19:0/3680622717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea881f0 0x55779eab1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 --2- 172.21.15.19:0/3680622717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea878d0 0x55779eaa7430 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 -- 172.21.15.19:0/3680622717 >> 172.21.15.19:0/3680622717 conn(0x55779ea83f60 msgr2=0x55779ea92480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 -- 172.21.15.19:0/3680622717 shutdown_connections 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.956+0000 7f5208f74740 1 -- 172.21.15.19:0/3680622717 wait complete. 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 1 -- start start 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 1 Processor -- start 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779eab1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 0x55779eaaae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779eaab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55779eaa8df0 con 0x55779ea96fc0 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55779eaa8720 con 0x55779ea881f0 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55779eaa94c0 con 0x55779ea878d0 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5208f74740 10 client.0 _ll_register_callbacks cb 0x7f51f80318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5205138640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779eaab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5205138640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779eaab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53018/0 (socket says 172.21.15.19:53018) 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5205138640 1 -- 172.21.15.19:0/1434317959 learned_addr learned my addr 172.21.15.19:0/1434317959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f51fffff640 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 0x55779eaaae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.957+0000 7f5205138640 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779eab1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5205138640 1 -- 172.21.15.19:0/1434317959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 msgr2=0x55779eab1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5205138640 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779eab1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5205138640 1 -- 172.21.15.19:0/1434317959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 msgr2=0x55779eaaae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5205138640 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 0x55779eaaae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5205138640 1 -- 172.21.15.19:0/1434317959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55779eaaf600 con 0x55779ea96fc0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5205138640 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779eaab550 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f51f8033010 tx=0x7f51f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180cf7da9694cefb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f51dffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f51dffff640 1 -- 172.21.15.19:0/1434317959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8063020 con 0x55779ea96fc0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f51dffff640 1 -- 172.21.15.19:0/1434317959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f51f804a7d0 con 0x55779ea96fc0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5208f74740 1 -- 172.21.15.19:0/1434317959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f8036070 con 0x55779ea96fc0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f51dffff640 1 -- 172.21.15.19:0/1434317959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f51f804a9d0 con 0x55779ea96fc0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5208f74740 10 client.15435 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f5208f74740 1 -- 172.21.15.19:0/1434317959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55779eb12390 con 0x55779ea96fc0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.958+0000 7f51dffff640 1 -- 172.21.15.19:0/1434317959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f8036070 con 0x55779ea96fc0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f51dffff640 1 -- 172.21.15.19:0/1434317959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f51f804c4f0 con 0x55779ea96fc0 2024-09-19T04:07:24.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f5208f74740 10 client.15435 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f5208f74740 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f5208f74740 20 client.15435 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f5208f74740 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55779eb16b60 0x55779eb15870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f5208f74740 4 client.15435 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f5208f74740 1 -- 172.21.15.19:0/1434317959 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "status"}) -- 0x55779e9f5b10 con 0x55779eb16b60 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f51fffff640 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55779eb16b60 0x55779eb15870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f51fffff640 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55779eb16b60 0x55779eb15870 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4be021afa033364 server_cookie=921a38ab269643d6 in_seq=0 out_seq=0 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.959+0000 7f51dffff640 10 client.15435 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f51dffff640 1 -- 172.21.15.19:0/1434317959 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55779e9f5b10 con 0x55779eb16b60 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f51dffff640 10 client.15435 handle_command_reply: tid=0 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 client.15435 shutdown 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 -- 172.21.15.19:0/1434317959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 msgr2=0x55779eaab550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779eaab550 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f51f8033010 tx=0x7f51f8009bd0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 -- 172.21.15.19:0/1434317959 shutdown_connections 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55779ea96fc0 0x55779eaab550 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55779ea881f0 0x55779eaaae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55779eb16b60 0x55779eb15870 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 --2- 172.21.15.19:0/1434317959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55779ea878d0 0x55779eab1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f51dffff640 0 client.15435 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 -- 172.21.15.19:0/1434317959 >> 172.21.15.19:0/1434317959 conn(0x55779ea83f60 msgr2=0x55779ea8ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 -- 172.21.15.19:0/1434317959 shutdown_connections 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 1 -- 172.21.15.19:0/1434317959 wait complete. 2024-09-19T04:07:24.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5208f74740 20 client.15435 trim_cache size 0 max 16384 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "0518de98-763b-11ef-bceb-c7b262605968", 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24476, 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 164.65646531799999, 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 29, 2024-09-19T04:07:24.972 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-09-19T04:07:24.973 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 93, 2024-09-19T04:07:24.973 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 175.199557301 2024-09-19T04:07:24.973 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:07:24.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5206b9b640 1 -- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51ec0a32d0 msgr2=0x7f51ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.960+0000 7f5206b9b640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51ec0a32d0 0x7f51ec0a5790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f51f0097d00 tx=0x7f51f0013500 comp rx=0 tx=0).stop 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 -- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200102d90 msgr2=0x7f52001985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200102d90 0x7f52001985b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f51f000b4a0 tx=0x7f51f000b960 comp rx=0 tx=0).stop 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 -- 172.21.15.19:0/2254647843 shutdown_connections 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f51ec0a32d0 0x7f51ec0a5790 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200108290 0x7f520019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5200103740 0x7f5200198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 --2- 172.21.15.19:0/2254647843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5200102d90 0x7f52001985b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 -- 172.21.15.19:0/2254647843 >> 172.21.15.19:0/2254647843 conn(0x7f52000fc7d0 msgr2=0x7f5200106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 -- 172.21.15.19:0/2254647843 shutdown_connections 2024-09-19T04:07:24.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:24.961+0000 7f5206b9b640 1 -- 172.21.15.19:0/2254647843 wait complete. 2024-09-19T04:07:25.879 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T04:07:26.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:26 smithi019 ceph-mon[30938]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-09-19T04:07:26.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:26 smithi111 ceph-mon[35365]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-09-19T04:07:26.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:26 smithi196 ceph-mon[35731]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-09-19T04:07:28.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:28 smithi111 ceph-mon[35365]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-09-19T04:07:28.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:28 smithi019 ceph-mon[30938]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-09-19T04:07:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:28 smithi196 ceph-mon[35731]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-09-19T04:07:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:30 smithi111 ceph-mon[35365]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-09-19T04:07:30.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:30 smithi019 ceph-mon[30938]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-09-19T04:07:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:30 smithi196 ceph-mon[35731]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 -- 172.21.15.19:0/385486944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9701025d0 msgr2=0x7fa9701049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 --2- 172.21.15.19:0/385486944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9701025d0 0x7fa9701049c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa960009900 tx=0x7fa96002a410 comp rx=0 tx=0).stop 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 -- 172.21.15.19:0/385486944 shutdown_connections 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 --2- 172.21.15.19:0/385486944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa970107830 0x7fa970109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 --2- 172.21.15.19:0/385486944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970104f00 0x7fa9701072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 --2- 172.21.15.19:0/385486944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9701025d0 0x7fa9701049c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 -- 172.21.15.19:0/385486944 >> 172.21.15.19:0/385486944 conn(0x7fa9700fc1b0 msgr2=0x7fa9700fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 -- 172.21.15.19:0/385486944 shutdown_connections 2024-09-19T04:07:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.651+0000 7fa975ea4640 1 -- 172.21.15.19:0/385486944 wait complete. 2024-09-19T04:07:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa975ea4640 1 Processor -- start 2024-09-19T04:07:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa975ea4640 1 -- start start 2024-09-19T04:07:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa975ea4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9701025d0 0x7fa970101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa975ea4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970104f00 0x7fa970100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa975ea4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa970107830 0x7fa970100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa975ea4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa97010d840 con 0x7fa970107830 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa975ea4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa97010d6c0 con 0x7fa970104f00 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa975ea4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa97010d9c0 con 0x7fa9701025d0 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.652+0000 7fa96ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa970107830 0x7fa970100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa970107830 0x7fa970100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53038/0 (socket says 172.21.15.19:53038) 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96ffff640 1 -- 172.21.15.19:0/2173013124 learned_addr learned my addr 172.21.15.19:0/2173013124 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9701025d0 0x7fa970101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96effd640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970104f00 0x7fa970100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96ffff640 1 -- 172.21.15.19:0/2173013124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9701025d0 msgr2=0x7fa970101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96ffff640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9701025d0 0x7fa970101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96ffff640 1 -- 172.21.15.19:0/2173013124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970104f00 msgr2=0x7fa970100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96ffff640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970104f00 0x7fa970100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96ffff640 1 -- 172.21.15.19:0/2173013124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa970101350 con 0x7fa970107830 2024-09-19T04:07:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96ffff640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa970107830 0x7fa970100ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa96800eb40 tx=0x7fa96800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96cff9640 1 -- 172.21.15.19:0/2173013124 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968004b20 con 0x7fa970107830 2024-09-19T04:07:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96cff9640 1 -- 172.21.15.19:0/2173013124 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa968004cc0 con 0x7fa970107830 2024-09-19T04:07:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa975ea4640 1 -- 172.21.15.19:0/2173013124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9701a1f60 con 0x7fa970107830 2024-09-19T04:07:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa96cff9640 1 -- 172.21.15.19:0/2173013124 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968010850 con 0x7fa970107830 2024-09-19T04:07:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.653+0000 7fa975ea4640 1 -- 172.21.15.19:0/2173013124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9701a23c0 con 0x7fa970107830 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa96cff9640 1 -- 172.21.15.19:0/2173013124 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fa968005020 con 0x7fa970107830 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa978c37740 1 Processor -- start 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa978c37740 1 -- start start 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa978c37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a09a0 0x556f761a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa978c37740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa978c37740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a6370 0x556f761a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa978c37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f761042f0 con 0x556f761a09a0 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa978c37740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556f760ac270 con 0x556f761a6370 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.656+0000 7fa978c37740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556f761002b0 con 0x556f761a12e0 2024-09-19T04:07:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a09a0 0x556f761a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96cff9640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa9580a3390 0x7fa9580a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a09a0 0x556f761a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53046/0 (socket says 172.21.15.19:53046) 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 -- 172.21.15.19:0/1152140137 learned_addr learned my addr 172.21.15.19:0/1152140137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/1152140137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96ffff640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa9580a3390 0x7fa9580a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 -- 172.21.15.19:0/1152140137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 msgr2=0x556f761a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 --2- 172.21.15.19:0/1152140137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 -- 172.21.15.19:0/1152140137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a6370 msgr2=0x556f761a8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 --2- 172.21.15.19:0/1152140137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a6370 0x556f761a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 -- 172.21.15.19:0/1152140137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f760ab340 con 0x556f761a09a0 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96cff9640 1 -- 172.21.15.19:0/2173013124 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fa9680d05a0 con 0x7fa970107830 2024-09-19T04:07:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96effd640 1 --2- 172.21.15.19:0/1152140137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a09a0 0x556f761a0da0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa964009d90 tx=0x7fa964013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af6f132c8cf3f77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa955ffb640 1 -- 172.21.15.19:0/1152140137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964004050 con 0x556f761a09a0 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa96ffff640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa9580a3390 0x7fa9580a5850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa96809c650 tx=0x7fa96806f540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa955ffb640 1 -- 172.21.15.19:0/1152140137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa964017030 con 0x556f761a09a0 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.657+0000 7fa955ffb640 1 -- 172.21.15.19:0/1152140137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964010430 con 0x556f761a09a0 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 -- 172.21.15.19:0/1152140137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a09a0 msgr2=0x556f761a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 --2- 172.21.15.19:0/1152140137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a09a0 0x556f761a0da0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa964009d90 tx=0x7fa964013830 comp rx=0 tx=0).stop 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 -- 172.21.15.19:0/1152140137 shutdown_connections 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 --2- 172.21.15.19:0/1152140137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a6370 0x556f761a8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 --2- 172.21.15.19:0/1152140137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 --2- 172.21.15.19:0/1152140137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a09a0 0x556f761a0da0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 -- 172.21.15.19:0/1152140137 >> 172.21.15.19:0/1152140137 conn(0x556f7619c010 msgr2=0x556f7619c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 -- 172.21.15.19:0/1152140137 shutdown_connections 2024-09-19T04:07:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 -- 172.21.15.19:0/1152140137 wait complete. 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.658+0000 7fa978c37740 1 -- start start 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa978c37740 1 Processor -- start 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa978c37740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f761aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa978c37740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa978c37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a6370 0x556f761b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa978c37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f761a9340 con 0x556f761a6370 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa978c37740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556f7608d290 con 0x556f761a09a0 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa978c37740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556f761a94c0 con 0x556f761a12e0 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa978c37740 10 client.0 _ll_register_callbacks cb 0x556f760be140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa96effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a6370 0x556f761b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa96effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a6370 0x556f761b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53062/0 (socket says 172.21.15.19:53062) 2024-09-19T04:07:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa96effd640 1 -- 172.21.15.19:0/1307485842 learned_addr learned my addr 172.21.15.19:0/1307485842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.659+0000 7fa96effd640 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f761aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa96effd640 1 -- 172.21.15.19:0/1307485842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 msgr2=0x556f761ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa96effd640 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa96effd640 1 -- 172.21.15.19:0/1307485842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 msgr2=0x556f761aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa96effd640 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f761aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa96effd640 1 -- 172.21.15.19:0/1307485842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f75e17210 con 0x556f761a6370 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa96effd640 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a6370 0x556f761b2120 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa9640037d0 tx=0x7fa964009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cef9d011a9b96ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa94effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa94effd640 1 -- 172.21.15.19:0/1307485842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964003930 con 0x556f761a6370 2024-09-19T04:07:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa94effd640 1 -- 172.21.15.19:0/1307485842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fa964004140 con 0x556f761a6370 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa978c37740 1 -- 172.21.15.19:0/1307485842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa964017030 con 0x556f761a6370 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa94effd640 1 -- 172.21.15.19:0/1307485842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa964010430 con 0x556f761a6370 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa978c37740 10 client.15447 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa978c37740 1 -- 172.21.15.19:0/1307485842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f75ee3620 con 0x556f761a6370 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa94effd640 1 -- 172.21.15.19:0/1307485842 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa964017030 con 0x556f761a6370 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.660+0000 7fa94effd640 1 -- 172.21.15.19:0/1307485842 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa96401c720 con 0x556f761a6370 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa978c37740 10 client.15447 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa978c37740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa978c37740 20 client.15447 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa978c37740 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556f761b6540 0x556f761b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa978c37740 4 client.15447 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa978c37740 1 -- 172.21.15.19:0/1307485842 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f76111a20 con 0x556f761b6540 2024-09-19T04:07:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556f761b6540 0x556f761b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556f761b6540 0x556f761b8930 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6decf42607ac016 server_cookie=59721b333ee899bc in_seq=0 out_seq=0 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.661+0000 7fa94effd640 10 client.15447 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa94effd640 1 -- 172.21.15.19:0/1307485842 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f76111a20 con 0x556f761b6540 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa94effd640 10 client.15447 handle_command_reply: tid=0 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 client.15447 shutdown 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 -- 172.21.15.19:0/1307485842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a6370 msgr2=0x556f761b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a6370 0x556f761b2120 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa9640037d0 tx=0x7fa964009e80 comp rx=0 tx=0).stop 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 -- 172.21.15.19:0/1307485842 shutdown_connections 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a6370 0x556f761b2120 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556f761b6540 0x556f761b8930 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a12e0 0x556f761ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 --2- 172.21.15.19:0/1307485842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f761aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa94effd640 0 client.15447 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 -- 172.21.15.19:0/1307485842 >> 172.21.15.19:0/1307485842 conn(0x556f7619c010 msgr2=0x556f7619d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 -- 172.21.15.19:0/1307485842 shutdown_connections 2024-09-19T04:07:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 1 -- 172.21.15.19:0/1307485842 wait complete. 2024-09-19T04:07:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.662+0000 7fa978c37740 20 client.15447 trim_cache size 0 max 16384 2024-09-19T04:07:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa978c37740 1 Processor -- start 2024-09-19T04:07:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa978c37740 1 -- start start 2024-09-19T04:07:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa978c37740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f7619b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa978c37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa978c37740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7619d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa978c37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ac9c0 con 0x556f761a12e0 2024-09-19T04:07:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa978c37740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ae500 con 0x556f761a09a0 2024-09-19T04:07:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa978c37740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ad090 con 0x556f761a6370 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53064/0 (socket says 172.21.15.19:53064) 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.677+0000 7fa96f7fe640 1 -- 172.21.15.19:0/186799121 learned_addr learned my addr 172.21.15.19:0/186799121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96effd640 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f7619b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96effd640 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7619d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96f7fe640 1 -- 172.21.15.19:0/186799121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 msgr2=0x556f7619d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7619d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96f7fe640 1 -- 172.21.15.19:0/186799121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 msgr2=0x556f7619b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f7619b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96f7fe640 1 -- 172.21.15.19:0/186799121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f761962f0 con 0x556f761a12e0 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96effd640 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7619d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:07:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761b2040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa9600314e0 tx=0x7fa960005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95aa2939567d82a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa9557fa640 1 -- 172.21.15.19:0/186799121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96004a070 con 0x556f761a12e0 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa9557fa640 1 -- 172.21.15.19:0/186799121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa960005520 con 0x556f761a12e0 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.678+0000 7fa9557fa640 1 -- 172.21.15.19:0/186799121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960035040 con 0x556f761a12e0 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 -- 172.21.15.19:0/186799121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 msgr2=0x556f761b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761b2040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa9600314e0 tx=0x7fa960005fd0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 -- 172.21.15.19:0/186799121 shutdown_connections 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7619d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761b2040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 --2- 172.21.15.19:0/186799121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f7619b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 -- 172.21.15.19:0/186799121 >> 172.21.15.19:0/186799121 conn(0x556f7619c010 msgr2=0x556f7619cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 -- 172.21.15.19:0/186799121 shutdown_connections 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 -- 172.21.15.19:0/186799121 wait complete. 2024-09-19T04:07:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 -- start start 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.679+0000 7fa978c37740 1 Processor -- start 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa978c37740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f761aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa978c37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa978c37740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7621f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa978c37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ade30 con 0x556f761a12e0 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa978c37740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ad760 con 0x556f761a09a0 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa978c37740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701aa9a0 con 0x556f761a6370 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa978c37740 10 client.0 _ll_register_callbacks cb 0x7fa964009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761af420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53066/0 (socket says 172.21.15.19:53066) 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 -- 172.21.15.19:0/544062148 learned_addr learned my addr 172.21.15.19:0/544062148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96effd640 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f761aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96effd640 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7621f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 -- 172.21.15.19:0/544062148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 msgr2=0x556f7621f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7621f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 -- 172.21.15.19:0/544062148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 msgr2=0x556f761aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f761aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 -- 172.21.15.19:0/544062148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7621f6d0 con 0x556f761a12e0 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa96f7fe640 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761af420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa960031440 tx=0x7fa960032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ee801b4c808aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa94effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa94effd640 1 -- 172.21.15.19:0/544062148 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96004a040 con 0x556f761a12e0 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa94effd640 1 -- 172.21.15.19:0/544062148 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fa960004ce0 con 0x556f761a12e0 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.680+0000 7fa978c37740 1 -- 172.21.15.19:0/544062148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa960005520 con 0x556f761a12e0 2024-09-19T04:07:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa94effd640 1 -- 172.21.15.19:0/544062148 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa960035040 con 0x556f761a12e0 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa978c37740 10 client.15453 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa978c37740 1 -- 172.21.15.19:0/544062148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f76220850 con 0x556f761a12e0 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa94effd640 1 -- 172.21.15.19:0/544062148 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa960005520 con 0x556f761a12e0 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa94effd640 1 -- 172.21.15.19:0/544062148 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa960056030 con 0x556f761a12e0 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa978c37740 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa978c37740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa978c37740 20 client.15453 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa978c37740 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556f76221250 0x556f76223670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa978c37740 4 client.15453 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa978c37740 1 -- 172.21.15.19:0/544062148 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f7610bd70 con 0x556f76221250 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.681+0000 7fa96effd640 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556f76221250 0x556f76223670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.682+0000 7fa96effd640 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556f76221250 0x556f76223670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3f0602544ec9b7 server_cookie=c2d570a78a5b9f42 in_seq=0 out_seq=0 2024-09-19T04:07:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.682+0000 7fa94effd640 10 client.15453 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.689+0000 7fa94effd640 1 -- 172.21.15.19:0/544062148 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f7610bd70 con 0x556f76221250 2024-09-19T04:07:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.689+0000 7fa94effd640 10 client.15453 handle_command_reply: tid=0 2024-09-19T04:07:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.689+0000 7fa978c37740 1 client.15453 shutdown 2024-09-19T04:07:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.689+0000 7fa978c37740 1 -- 172.21.15.19:0/544062148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 msgr2=0x556f761af420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.689+0000 7fa978c37740 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761af420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa960031440 tx=0x7fa960032f80 comp rx=0 tx=0).stop 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 1 -- 172.21.15.19:0/544062148 shutdown_connections 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556f76221250 0x556f76223670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f761a6370 0x556f7621f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f761a12e0 0x556f761af420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa94effd640 0 client.15453 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 1 --2- 172.21.15.19:0/544062148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f761a09a0 0x556f761aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 1 -- 172.21.15.19:0/544062148 >> 172.21.15.19:0/544062148 conn(0x556f7619c010 msgr2=0x556f7619d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 1 -- 172.21.15.19:0/544062148 shutdown_connections 2024-09-19T04:07:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 1 -- 172.21.15.19:0/544062148 wait complete. 2024-09-19T04:07:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa978c37740 20 client.15453 trim_cache size 0 max 16384 2024-09-19T04:07:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:07:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:07:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:07:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:07:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 -- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa9580a3390 msgr2=0x7fa9580a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa9580a3390 0x7fa9580a5850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa96809c650 tx=0x7fa96806f540 comp rx=0 tx=0).stop 2024-09-19T04:07:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 -- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa970107830 msgr2=0x7fa970100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa970107830 0x7fa970100ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa96800eb40 tx=0x7fa96800c2d0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 -- 172.21.15.19:0/2173013124 shutdown_connections 2024-09-19T04:07:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa9580a3390 0x7fa9580a5850 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa970107830 0x7fa970100ac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970104f00 0x7fa970100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 --2- 172.21.15.19:0/2173013124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9701025d0 0x7fa970101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.690+0000 7fa975ea4640 1 -- 172.21.15.19:0/2173013124 >> 172.21.15.19:0/2173013124 conn(0x7fa9700fc1b0 msgr2=0x7fa970102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.691+0000 7fa975ea4640 1 -- 172.21.15.19:0/2173013124 shutdown_connections 2024-09-19T04:07:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:30.691+0000 7fa975ea4640 1 -- 172.21.15.19:0/2173013124 wait complete. 2024-09-19T04:07:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:32 smithi111 ceph-mon[35365]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:32.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:32 smithi019 ceph-mon[30938]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:32 smithi196 ceph-mon[35731]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:34 smithi111 ceph-mon[35365]: pgmap v379: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T04:07:34.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:34 smithi019 ceph-mon[30938]: pgmap v379: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T04:07:34.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:34 smithi196 ceph-mon[35731]: pgmap v379: 129 pgs: 129 active+clean; 665 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T04:07:36.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:36 smithi111 ceph-mon[35365]: pgmap v380: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:36.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:36 smithi019 ceph-mon[30938]: pgmap v380: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:36 smithi196 ceph-mon[35731]: pgmap v380: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:38.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:38 smithi111 ceph-mon[35365]: pgmap v381: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:38.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:38 smithi019 ceph-mon[30938]: pgmap v381: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:38.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:38 smithi196 ceph-mon[35731]: pgmap v381: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:07:40.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:40 smithi111 ceph-mon[35365]: pgmap v382: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-19T04:07:40.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:40 smithi019 ceph-mon[30938]: pgmap v382: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-19T04:07:40.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:40 smithi196 ceph-mon[35731]: pgmap v382: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-19T04:07:40.880 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T04:07:40.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:07:41.147 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 -- 172.21.15.19:0/2212581803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c100e20 msgr2=0x7fc67c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 --2- 172.21.15.19:0/2212581803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c100e20 0x7fc67c101200 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009900 tx=0x7fc66c02a480 comp rx=0 tx=0).stop 2024-09-19T04:07:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 -- 172.21.15.19:0/2212581803 shutdown_connections 2024-09-19T04:07:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 --2- 172.21.15.19:0/2212581803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67c102ef0 0x7fc67c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 --2- 172.21.15.19:0/2212581803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c102570 0x7fc67c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 --2- 172.21.15.19:0/2212581803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c100e20 0x7fc67c101200 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 -- 172.21.15.19:0/2212581803 >> 172.21.15.19:0/2212581803 conn(0x7fc67c0fc7d0 msgr2=0x7fc67c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 -- 172.21.15.19:0/2212581803 shutdown_connections 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.648+0000 7fc68114b640 1 -- 172.21.15.19:0/2212581803 wait complete. 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.649+0000 7fc68114b640 1 Processor -- start 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.649+0000 7fc68114b640 1 -- start start 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc68114b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67c100e20 0x7fc67c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc68114b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c102570 0x7fc67c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc68114b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c102ef0 0x7fc67c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc68114b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c110f80 con 0x7fc67c102570 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67c100e20 0x7fc67c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67c100e20 0x7fc67c197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:44848/0 (socket says 172.21.15.19:44848) 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc68114b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c110e00 con 0x7fc67c100e20 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c111100 con 0x7fc67c102ef0 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67ad9d640 1 -- 172.21.15.19:0/1824169858 learned_addr learned my addr 172.21.15.19:0/1824169858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67a59c640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c102570 0x7fc67c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67b59e640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c102ef0 0x7fc67c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67ad9d640 1 -- 172.21.15.19:0/1824169858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c102ef0 msgr2=0x7fc67c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67ad9d640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c102ef0 0x7fc67c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67ad9d640 1 -- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c102570 msgr2=0x7fc67c197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67ad9d640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c102570 0x7fc67c197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67ad9d640 1 -- 172.21.15.19:0/1824169858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c1a6980 con 0x7fc67c100e20 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.650+0000 7fc67a59c640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c102570 0x7fc67c197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.651+0000 7fc67ad9d640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67c100e20 0x7fc67c197110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc66c006010 tx=0x7fc66c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.651+0000 7fc663fff640 1 -- 172.21.15.19:0/1824169858 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c03f070 con 0x7fc67c100e20 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.651+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67c1a6c10 con 0x7fc67c100e20 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.651+0000 7fc663fff640 1 -- 172.21.15.19:0/1824169858 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c031c30 con 0x7fc67c100e20 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.651+0000 7fc663fff640 1 -- 172.21.15.19:0/1824169858 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c035030 con 0x7fc67c100e20 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.651+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67c1a70f0 con 0x7fc67c100e20 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.652+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc67c101200 con 0x7fc67c100e20 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.654+0000 7fc663fff640 1 -- 172.21.15.19:0/1824169858 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fc66c042070 con 0x7fc67c100e20 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.655+0000 7fc663fff640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6640a32d0 0x7fc6640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.655+0000 7fc67a59c640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6640a32d0 0x7fc6640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.655+0000 7fc663fff640 1 -- 172.21.15.19:0/1824169858 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fc66c0f0a00 con 0x7fc67c100e20 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.656+0000 7fc67a59c640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6640a32d0 0x7fc6640a5790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc67c199270 tx=0x7fc670005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.658+0000 7fc663fff640 1 -- 172.21.15.19:0/1824169858 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc66c031de0 con 0x7fc67c100e20 2024-09-19T04:07:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.898+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc67c06a5a0 con 0x7fc67c100e20 2024-09-19T04:07:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.899+0000 7fc663fff640 1 -- 172.21.15.19:0/1824169858 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7fc67c06a5a0 con 0x7fc67c100e20 2024-09-19T04:07:41.901 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:41.901 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:07:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6640a32d0 msgr2=0x7fc6640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6640a32d0 0x7fc6640a5790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc67c199270 tx=0x7fc670005f90 comp rx=0 tx=0).stop 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67c100e20 msgr2=0x7fc67c197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67c100e20 0x7fc67c197110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc66c006010 tx=0x7fc66c003c10 comp rx=0 tx=0).stop 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 shutdown_connections 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc6640a32d0 0x7fc6640a5790 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c102ef0 0x7fc67c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc67c102570 0x7fc67c197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 --2- 172.21.15.19:0/1824169858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67c100e20 0x7fc67c197110 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.902+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 >> 172.21.15.19:0/1824169858 conn(0x7fc67c0fc7d0 msgr2=0x7fc67c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.903+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 shutdown_connections 2024-09-19T04:07:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:41.903+0000 7fc68114b640 1 -- 172.21.15.19:0/1824169858 wait complete. 2024-09-19T04:07:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:42 smithi019 ceph-mon[30938]: pgmap v383: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T04:07:42.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:42 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1824169858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:42.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:42 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:42.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:42 smithi111 ceph-mon[35365]: pgmap v383: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T04:07:42.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:42.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:42 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1824169858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:42.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:42.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:42 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:42.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:42 smithi196 ceph-mon[35731]: pgmap v383: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T04:07:42.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:07:42.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:42 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1824169858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:42.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:42.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:42 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:42.826 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T04:07:42.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T04:07:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 -- 172.21.15.19:0/2667673388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc001025f0 msgr2=0x7fcc00108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 --2- 172.21.15.19:0/2667673388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc001025f0 0x7fcc00108e80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009900 tx=0x7fcbf802a410 comp rx=0 tx=0).stop 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 -- 172.21.15.19:0/2667673388 shutdown_connections 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 --2- 172.21.15.19:0/2667673388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc001025f0 0x7fcc00108e80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 --2- 172.21.15.19:0/2667673388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc00101c30 0x7fcc001020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 --2- 172.21.15.19:0/2667673388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc00100a40 0x7fcc00100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 -- 172.21.15.19:0/2667673388 >> 172.21.15.19:0/2667673388 conn(0x7fcc000fc1e0 msgr2=0x7fcc000fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 -- 172.21.15.19:0/2667673388 shutdown_connections 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 -- 172.21.15.19:0/2667673388 wait complete. 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.782+0000 7fcc0599f640 1 Processor -- start 2024-09-19T04:07:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcc0599f640 1 -- start start 2024-09-19T04:07:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcc0599f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc00100a40 0x7fcc0006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcc0599f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc00101c30 0x7fcc0019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcc0599f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc001025f0 0x7fcc001a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcc0599f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010c6c0 con 0x7fcc00100a40 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcc0599f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010c540 con 0x7fcc001025f0 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcc0599f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010c840 con 0x7fcc00101c30 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcbfeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc00100a40 0x7fcc0006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcbfeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc00100a40 0x7fcc0006a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43696/0 (socket says 172.21.15.19:43696) 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcbfeffd640 1 -- 172.21.15.19:0/3531365296 learned_addr learned my addr 172.21.15.19:0/3531365296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc001025f0 0x7fcc001a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.783+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc00101c30 0x7fcc0019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcbfeffd640 1 -- 172.21.15.19:0/3531365296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc00101c30 msgr2=0x7fcc0019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc00101c30 0x7fcc0019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcbfeffd640 1 -- 172.21.15.19:0/3531365296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc001025f0 msgr2=0x7fcc001a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc001025f0 0x7fcc001a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcbfeffd640 1 -- 172.21.15.19:0/3531365296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc001a1c20 con 0x7fcc00100a40 2024-09-19T04:07:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc00100a40 0x7fcc0006a410 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000eb70 tx=0x7fcbf000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcc0499d640 1 -- 172.21.15.19:0/3531365296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0003b70 con 0x7fcc00100a40 2024-09-19T04:07:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcc0499d640 1 -- 172.21.15.19:0/3531365296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf000ee70 con 0x7fcc00100a40 2024-09-19T04:07:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcc0599f640 1 -- 172.21.15.19:0/3531365296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc001a1eb0 con 0x7fcc00100a40 2024-09-19T04:07:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcc0499d640 1 -- 172.21.15.19:0/3531365296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0010830 con 0x7fcc00100a40 2024-09-19T04:07:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.784+0000 7fcc0599f640 1 -- 172.21.15.19:0/3531365296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001a2310 con 0x7fcc00100a40 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0881c740 1 Processor -- start 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0881c740 1 -- start start 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29ca5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0881c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb29c012f0 con 0x55bb29ca3370 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0881c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bb29ba9270 con 0x55bb29c9e2e0 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0881c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb29bfd2b0 con 0x55bb29c9d9a0 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbfe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29ca5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbfe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29ca5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43706/0 (socket says 172.21.15.19:43706) 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbfe7fc640 1 -- 172.21.15.19:0/3841283501 learned_addr learned my addr 172.21.15.19:0/3841283501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/3841283501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcc0499d640 1 -- 172.21.15.19:0/3531365296 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fcbf0004020 con 0x7fcc00100a40 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbfe7fc640 1 -- 172.21.15.19:0/3841283501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 msgr2=0x55bb29c9dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/3841283501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbfe7fc640 1 -- 172.21.15.19:0/3841283501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 msgr2=0x55bb29ca2da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/3841283501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca2da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.787+0000 7fcbfe7fc640 1 -- 172.21.15.19:0/3841283501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb29ba8340 con 0x55bb29ca3370 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.788+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/3841283501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29ca5830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800bc80 tx=0x7fcbe800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16441b8049c30d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.788+0000 7fcbf5ffb640 1 -- 172.21.15.19:0/3841283501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8015070 con 0x55bb29ca3370 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.788+0000 7fcbf5ffb640 1 -- 172.21.15.19:0/3841283501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe801a940 con 0x55bb29ca3370 2024-09-19T04:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.788+0000 7fcbf5ffb640 1 -- 172.21.15.19:0/3841283501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8011a10 con 0x55bb29ca3370 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.788+0000 7fcc0499d640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcbec0a33b0 0x7fcbec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.788+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcbec0a33b0 0x7fcbec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.788+0000 7fcc0499d640 1 -- 172.21.15.19:0/3531365296 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fcbf00d04f0 con 0x7fcc00100a40 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 -- 172.21.15.19:0/3841283501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 msgr2=0x55bb29ca5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 --2- 172.21.15.19:0/3841283501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29ca5830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800bc80 tx=0x7fcbe800f730 comp rx=0 tx=0).stop 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 -- 172.21.15.19:0/3841283501 shutdown_connections 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 --2- 172.21.15.19:0/3841283501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29ca5830 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 --2- 172.21.15.19:0/3841283501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 --2- 172.21.15.19:0/3841283501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 -- 172.21.15.19:0/3841283501 >> 172.21.15.19:0/3841283501 conn(0x55bb29c99010 msgr2=0x55bb29c99410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 -- 172.21.15.19:0/3841283501 shutdown_connections 2024-09-19T04:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 -- 172.21.15.19:0/3841283501 wait complete. 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcbec0a33b0 0x7fcbec0a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8006010 tx=0x7fcbf80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.789+0000 7fcc0881c740 1 -- start start 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcc0881c740 1 Processor -- start 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29ca7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29ca3370 0x55bb29caf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcc0881c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb29ca6340 con 0x55bb29c9e2e0 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcc0881c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bb29b8a290 con 0x55bb29ca3370 2024-09-19T04:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcc0881c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb29ca64c0 con 0x55bb29c9d9a0 2024-09-19T04:07:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcc0881c740 10 client.0 _ll_register_callbacks cb 0x55bb29bbb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43712/0 (socket says 172.21.15.19:43712) 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfe7fc640 1 -- 172.21.15.19:0/271298023 learned_addr learned my addr 172.21.15.19:0/271298023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29ca7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29ca3370 0x55bb29caf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfe7fc640 1 -- 172.21.15.19:0/271298023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 msgr2=0x55bb29ca7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29ca7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfe7fc640 1 -- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29ca3370 msgr2=0x55bb29caf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29ca3370 0x55bb29caf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbfe7fc640 1 -- 172.21.15.19:0/271298023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb29914210 con 0x55bb29c9e2e0 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.790+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29ca3370 0x55bb29caf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca8230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8006010 tx=0x7fcbe800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0d6d093ab02f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcbdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcbdeffd640 1 -- 172.21.15.19:0/271298023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8015070 con 0x55bb29c9e2e0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcbdeffd640 1 -- 172.21.15.19:0/271298023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fcbe801aaf0 con 0x55bb29c9e2e0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 1 -- 172.21.15.19:0/271298023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe801a940 con 0x55bb29c9e2e0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcbdeffd640 1 -- 172.21.15.19:0/271298023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbe801a770 con 0x55bb29c9e2e0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 1 -- 172.21.15.19:0/271298023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb299e0620 con 0x55bb29c9e2e0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcbdeffd640 1 -- 172.21.15.19:0/271298023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe801a940 con 0x55bb29c9e2e0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcbdeffd640 1 -- 172.21.15.19:0/271298023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcbe8011ab0 con 0x55bb29c9e2e0 2024-09-19T04:07:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 20 client.15471 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bb29cb3740 0x55bb29cb5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 4 client.15471 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.791+0000 7fcc0881c740 1 -- 172.21.15.19:0/271298023 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb29c0ea20 con 0x55bb29cb3740 2024-09-19T04:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.792+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bb29cb3740 0x55bb29cb5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.792+0000 7fcbdeffd640 10 client.15471 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.792+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bb29cb3740 0x55bb29cb5b30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d07b66c96a6b49 server_cookie=5d2790f82ca0777c in_seq=0 out_seq=0 2024-09-19T04:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcbdeffd640 1 -- 172.21.15.19:0/271298023 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb29c0ea20 con 0x55bb29cb3740 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcbdeffd640 10 client.15471 handle_command_reply: tid=0 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 client.15471 shutdown 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 -- 172.21.15.19:0/271298023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9e2e0 msgr2=0x55bb29ca8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca8230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8006010 tx=0x7fcbe800bfa0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 -- 172.21.15.19:0/271298023 shutdown_connections 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29ca3370 0x55bb29caf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9e2e0 0x55bb29ca8230 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bb29cb3740 0x55bb29cb5b30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 --2- 172.21.15.19:0/271298023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29ca7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 -- 172.21.15.19:0/271298023 >> 172.21.15.19:0/271298023 conn(0x55bb29c99010 msgr2=0x55bb29c9a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcbdeffd640 0 client.15471 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 -- 172.21.15.19:0/271298023 shutdown_connections 2024-09-19T04:07:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 1 -- 172.21.15.19:0/271298023 wait complete. 2024-09-19T04:07:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.793+0000 7fcc0881c740 20 client.15471 trim_cache size 0 max 16384 2024-09-19T04:07:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcc0881c740 1 Processor -- start 2024-09-19T04:07:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcc0881c740 1 -- start start 2024-09-19T04:07:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29caf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29ca3370 0x55bb29c9a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcc0881c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001ac900 con 0x55bb29c9d9a0 2024-09-19T04:07:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcc0881c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001ae440 con 0x55bb29c9e2e0 2024-09-19T04:07:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcc0881c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001acfd0 con 0x55bb29ca3370 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43722/0 (socket says 172.21.15.19:43722) 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 -- 172.21.15.19:0/648033106 learned_addr learned my addr 172.21.15.19:0/648033106 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29ca3370 0x55bb29c9a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29caf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 -- 172.21.15.19:0/648033106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29ca3370 msgr2=0x55bb29c9a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29ca3370 0x55bb29c9a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 -- 172.21.15.19:0/648033106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 msgr2=0x55bb29caf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29caf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 -- 172.21.15.19:0/648033106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb29c98e30 con 0x55bb29c9d9a0 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29ca3370 0x55bb29c9a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.823+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9a400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000cba0 tx=0x7fcbf00d07b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfcc941320f7244 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcbf57fa640 1 -- 172.21.15.19:0/648033106 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf009cd10 con 0x55bb29c9d9a0 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcbf57fa640 1 -- 172.21.15.19:0/648033106 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf00b5070 con 0x55bb29c9d9a0 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcbf57fa640 1 -- 172.21.15.19:0/648033106 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00cdd50 con 0x55bb29c9d9a0 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcc0881c740 1 -- 172.21.15.19:0/648033106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9d9a0 msgr2=0x55bb29c9a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcc0881c740 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9a400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000cba0 tx=0x7fcbf00d07b0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcc0881c740 1 -- 172.21.15.19:0/648033106 shutdown_connections 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcc0881c740 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29ca3370 0x55bb29c9a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcc0881c740 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29caf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcc0881c740 1 --2- 172.21.15.19:0/648033106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29c9d9a0 0x55bb29c9a400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcc0881c740 1 -- 172.21.15.19:0/648033106 >> 172.21.15.19:0/648033106 conn(0x55bb29c99010 msgr2=0x55bb29c981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.824+0000 7fcc0881c740 1 -- 172.21.15.19:0/648033106 shutdown_connections 2024-09-19T04:07:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 -- 172.21.15.19:0/648033106 wait complete. 2024-09-19T04:07:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 -- start start 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 Processor -- start 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29cac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29caceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29cadec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001add70 con 0x55bb29ca3370 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001ad6a0 con 0x55bb29c9e2e0 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.825+0000 7fcc0881c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001aa8e0 con 0x55bb29c9d9a0 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcc0881c740 10 client.0 _ll_register_callbacks cb 0x7fcbf8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29cadec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29cadec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43730/0 (socket says 172.21.15.19:43730) 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 -- 172.21.15.19:0/2526527610 learned_addr learned my addr 172.21.15.19:0/2526527610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbfe7fc640 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29caceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29cac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 -- 172.21.15.19:0/2526527610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 msgr2=0x55bb29cac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29cac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 -- 172.21.15.19:0/2526527610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 msgr2=0x55bb29caceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29caceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 -- 172.21.15.19:0/2526527610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb29cae4d0 con 0x55bb29ca3370 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbff7fe640 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29cadec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009900 tx=0x7fcbf8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18884758d3e7858b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbdeffd640 1 -- 172.21.15.19:0/2526527610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8035040 con 0x55bb29ca3370 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcbdeffd640 1 -- 172.21.15.19:0/2526527610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fcbf8003ac0 con 0x55bb29ca3370 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.826+0000 7fcc0881c740 1 -- 172.21.15.19:0/2526527610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf00b5070 con 0x55bb29ca3370 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcbdeffd640 1 -- 172.21.15.19:0/2526527610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbf8003cc0 con 0x55bb29ca3370 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcc0881c740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcc0881c740 1 -- 172.21.15.19:0/2526527610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb29d1e490 con 0x55bb29ca3370 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcbdeffd640 1 -- 172.21.15.19:0/2526527610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf00b5070 con 0x55bb29ca3370 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcbdeffd640 1 -- 172.21.15.19:0/2526527610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcbf803d6b0 con 0x55bb29ca3370 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcc0881c740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcc0881c740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:07:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcc0881c740 20 client.15477 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcc0881c740 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bb29d1ee40 0x55bb29d21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcc0881c740 4 client.15477 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:07:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcc0881c740 1 -- 172.21.15.19:0/2526527610 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb29c08d70 con 0x55bb29d1ee40 2024-09-19T04:07:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.827+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bb29d1ee40 0x55bb29d21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.828+0000 7fcbdeffd640 10 client.15477 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.828+0000 7fcbfeffd640 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bb29d1ee40 0x55bb29d21260 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec5c72b2a16e43ae server_cookie=65670b6c0aa3417a in_seq=0 out_seq=0 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcbdeffd640 1 -- 172.21.15.19:0/2526527610 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb29c08d70 con 0x55bb29d1ee40 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcbdeffd640 10 client.15477 handle_command_reply: tid=0 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 client.15477 shutdown 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 -- 172.21.15.19:0/2526527610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 msgr2=0x55bb29cadec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29cadec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009900 tx=0x7fcbf8009260 comp rx=0 tx=0).stop 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 -- 172.21.15.19:0/2526527610 shutdown_connections 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb29ca3370 0x55bb29cadec0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb29c9e2e0 0x55bb29caceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bb29d1ee40 0x55bb29d21260 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 --2- 172.21.15.19:0/2526527610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb29c9d9a0 0x55bb29cac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 -- 172.21.15.19:0/2526527610 >> 172.21.15.19:0/2526527610 conn(0x55bb29c99010 msgr2=0x55bb29c993f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcbdeffd640 0 client.15477 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 -- 172.21.15.19:0/2526527610 shutdown_connections 2024-09-19T04:07:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 1 -- 172.21.15.19:0/2526527610 wait complete. 2024-09-19T04:07:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.834+0000 7fcc0881c740 20 client.15477 trim_cache size 0 max 16384 2024-09-19T04:07:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:07:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:07:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:07:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:07:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 -- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcbec0a33b0 msgr2=0x7fcbec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcbec0a33b0 0x7fcbec0a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8006010 tx=0x7fcbf80315b0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 -- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc00100a40 msgr2=0x7fcc0006a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc00100a40 0x7fcc0006a410 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000eb70 tx=0x7fcbf000c6a0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 -- 172.21.15.19:0/3531365296 shutdown_connections 2024-09-19T04:07:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcbec0a33b0 0x7fcbec0a5870 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc001025f0 0x7fcc001a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc00101c30 0x7fcc0019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 --2- 172.21.15.19:0/3531365296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc00100a40 0x7fcc0006a410 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 -- 172.21.15.19:0/3531365296 >> 172.21.15.19:0/3531365296 conn(0x7fcc000fc1e0 msgr2=0x7fcc00104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 -- 172.21.15.19:0/3531365296 shutdown_connections 2024-09-19T04:07:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:43.835+0000 7fcc0599f640 1 -- 172.21.15.19:0/3531365296 wait complete. 2024-09-19T04:07:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:44 smithi111 ceph-mon[35365]: pgmap v384: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T04:07:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:44 smithi019 ceph-mon[30938]: pgmap v384: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T04:07:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:44 smithi196 ceph-mon[35731]: pgmap v384: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T04:07:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:07:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:07:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:07:46.284 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:46 smithi019 ceph-mon[30938]: pgmap v385: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T04:07:46.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:46 smithi111 ceph-mon[35365]: pgmap v385: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T04:07:46.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:46 smithi196 ceph-mon[35731]: pgmap v385: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 -- 172.21.15.19:0/2749458402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc108300 msgr2=0x7f17dc10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 --2- 172.21.15.19:0/2749458402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc108300 0x7f17dc10ec70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009cd0 tx=0x7f17d402a8d0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 -- 172.21.15.19:0/2749458402 shutdown_connections 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 --2- 172.21.15.19:0/2749458402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc108300 0x7f17dc10ec70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 --2- 172.21.15.19:0/2749458402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17dc103780 0x7f17dc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 --2- 172.21.15.19:0/2749458402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17dc102dd0 0x7f17dc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 -- 172.21.15.19:0/2749458402 >> 172.21.15.19:0/2749458402 conn(0x7f17dc0fc7d0 msgr2=0x7f17dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 -- 172.21.15.19:0/2749458402 shutdown_connections 2024-09-19T04:07:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.243+0000 7f17e4752640 1 -- 172.21.15.19:0/2749458402 wait complete. 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.244+0000 7f17e4752640 1 Processor -- start 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.244+0000 7f17e4752640 1 -- start start 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.244+0000 7f17e4752640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17dc102dd0 0x7f17dc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.244+0000 7f17e4752640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc103780 0x7f17dc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.244+0000 7f17e4752640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17dc108300 0x7f17dc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.244+0000 7f17e4752640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc112120 con 0x7f17dc103780 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.244+0000 7f17e4752640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc111fa0 con 0x7f17dc108300 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.244+0000 7f17e4752640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1122a0 con 0x7f17dc102dd0 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17dc102dd0 0x7f17dc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17dc102dd0 0x7f17dc198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:42236/0 (socket says 172.21.15.19:42236) 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 -- 172.21.15.19:0/2467454351 learned_addr learned my addr 172.21.15.19:0/2467454351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e2cef640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17dc108300 0x7f17dc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e1ced640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc103780 0x7f17dc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 -- 172.21.15.19:0/2467454351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17dc108300 msgr2=0x7f17dc19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17dc108300 0x7f17dc19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 -- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc103780 msgr2=0x7f17dc198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc103780 0x7f17dc198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 -- 172.21.15.19:0/2467454351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc19d530 con 0x7f17dc102dd0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e1ced640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc103780 0x7f17dc198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e2cef640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17dc108300 0x7f17dc19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.245+0000 7f17e24ee640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17dc102dd0 0x7f17dc198580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f17d800d950 tx=0x7f17d800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.246+0000 7f17d37fe640 1 -- 172.21.15.19:0/2467454351 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8017070 con 0x7f17dc102dd0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.246+0000 7f17d37fe640 1 -- 172.21.15.19:0/2467454351 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d8009c30 con 0x7f17dc102dd0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.246+0000 7f17d37fe640 1 -- 172.21.15.19:0/2467454351 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d800f6b0 con 0x7f17dc102dd0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.246+0000 7f17e4752640 1 -- 172.21.15.19:0/2467454351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc19d820 con 0x7f17dc102dd0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.246+0000 7f17e4752640 1 -- 172.21.15.19:0/2467454351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1a5100 con 0x7f17dc102dd0 2024-09-19T04:07:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.249+0000 7f17d37fe640 1 -- 172.21.15.19:0/2467454351 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f17d80109d0 con 0x7f17dc102dd0 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.249+0000 7f17d37fe640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f17c80a32d0 0x7f17c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.249+0000 7f17e6b2b740 1 Processor -- start 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17d37fe640 1 -- 172.21.15.19:0/2467454351 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f17d80cfb10 con 0x7f17dc102dd0 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e1ced640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f17c80a32d0 0x7f17c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e6b2b740 1 -- start start 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e62b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e6fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c462dc82a0 con 0x55c462e62760 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c462d6fb10 con 0x55c462e63080 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c462dc4260 con 0x55c462e6d940 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e24ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e62b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e24ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e62b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43752/0 (socket says 172.21.15.19:43752) 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e24ee640 1 -- 172.21.15.19:0/1167717520 learned_addr learned my addr 172.21.15.19:0/1167717520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.250+0000 7f17e2cef640 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e6fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17e1ced640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f17c80a32d0 0x7f17c80a5790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005fd0 tx=0x7f17cc00b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17e1ced640 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17e24ee640 1 -- 172.21.15.19:0/1167717520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 msgr2=0x55c462e6fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17e24ee640 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e6fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17e24ee640 1 -- 172.21.15.19:0/1167717520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 msgr2=0x55c462e6d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17e24ee640 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17e24ee640 1 -- 172.21.15.19:0/1167717520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c462b39b90 con 0x55c462e62760 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17e24ee640 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e62b40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f17d8010fd0 tx=0x7f17d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbdf60323cf643f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17d0ff9640 1 -- 172.21.15.19:0/1167717520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8020040 con 0x55c462e62760 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17d0ff9640 1 -- 172.21.15.19:0/1167717520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d809ff80 con 0x55c462e62760 2024-09-19T04:07:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.251+0000 7f17d0ff9640 1 -- 172.21.15.19:0/1167717520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d809b280 con 0x55c462e62760 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.252+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1167717520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 msgr2=0x55c462e62b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.252+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e62b40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f17d8010fd0 tx=0x7f17d8005880 comp rx=0 tx=0).stop 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1167717520 shutdown_connections 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e6fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/1167717520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e62b40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1167717520 >> 172.21.15.19:0/1167717520 conn(0x55c462e5ecf0 msgr2=0x55c462e5f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1167717520 shutdown_connections 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1167717520 wait complete. 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 -- start start 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.253+0000 7f17e6b2b740 1 Processor -- start 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e6a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e6d940 0x55c462e6b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c462e704b0 con 0x55c462e6d940 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c462d506e0 con 0x55c462e63080 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c462e70630 con 0x55c462e62760 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e6b2b740 10 client.0 _ll_register_callbacks cb 0x55c462d74a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e2cef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e6d940 0x55c462e6b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e2cef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e6d940 0x55c462e6b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43762/0 (socket says 172.21.15.19:43762) 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e2cef640 1 -- 172.21.15.19:0/904465387 learned_addr learned my addr 172.21.15.19:0/904465387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.254+0000 7f17e1ced640 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e1ced640 1 -- 172.21.15.19:0/904465387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 msgr2=0x55c462e6a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e24ee640 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e6a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e1ced640 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e6a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e1ced640 1 -- 172.21.15.19:0/904465387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e6d940 msgr2=0x55c462e6b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e1ced640 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e6d940 0x55c462e6b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e1ced640 1 -- 172.21.15.19:0/904465387 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c462e7de30 con 0x55c462e63080 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e2cef640 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e6d940 0x55c462e6b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e1ced640 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6ade0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17dc06a2d0 tx=0x7f17cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da5e9e920a81fcaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17c5ffb640 1 -- 172.21.15.19:0/904465387 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc01f9c0 con 0x55c462e63080 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17e6b2b740 1 -- 172.21.15.19:0/904465387 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d809ff80 con 0x55c462e63080 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.255+0000 7f17c5ffb640 1 -- 172.21.15.19:0/904465387 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f17cc01fb60 con 0x55c462e63080 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.256+0000 7f17c5ffb640 1 -- 172.21.15.19:0/904465387 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17cc01fda0 con 0x55c462e63080 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.256+0000 7f17e6b2b740 10 client.25106 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.256+0000 7f17e6b2b740 1 -- 172.21.15.19:0/904465387 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c462e80870 con 0x55c462e63080 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.256+0000 7f17c5ffb640 1 -- 172.21.15.19:0/904465387 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d809ff80 con 0x55c462e63080 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.256+0000 7f17c5ffb640 1 -- 172.21.15.19:0/904465387 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f17cc01e450 con 0x55c462e63080 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.256+0000 7f17e6b2b740 10 client.25106 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.256+0000 7f17e6b2b740 10 client.25106 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:07:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.257+0000 7f17e6b2b740 20 client.25106 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.257+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c462e81c80 0x55c462e840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.257+0000 7f17e6b2b740 4 client.25106 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.257+0000 7f17e6b2b740 1 -- 172.21.15.19:0/904465387 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c462dd59d0 con 0x55c462e81c80 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.257+0000 7f17e24ee640 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c462e81c80 0x55c462e840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.257+0000 7f17e24ee640 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c462e81c80 0x55c462e840d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd9994c6e59ce1e server_cookie=58a3397ba59a735d in_seq=0 out_seq=0 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.257+0000 7f17c5ffb640 10 client.25106 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17c5ffb640 1 -- 172.21.15.19:0/904465387 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c462dd59d0 con 0x55c462e81c80 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17c5ffb640 10 client.25106 handle_command_reply: tid=0 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 client.25106 shutdown 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 -- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 msgr2=0x55c462e6ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6ade0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17dc06a2d0 tx=0x7f17cc0046c0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 -- 172.21.15.19:0/904465387 shutdown_connections 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e6d940 0x55c462e6b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e6ade0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c462e81c80 0x55c462e840d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/904465387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e6a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17c5ffb640 0 client.25106 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 -- 172.21.15.19:0/904465387 >> 172.21.15.19:0/904465387 conn(0x55c462e5ecf0 msgr2=0x55c462e5de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 -- 172.21.15.19:0/904465387 shutdown_connections 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.258+0000 7f17e6b2b740 1 -- 172.21.15.19:0/904465387 wait complete. 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.259+0000 7f17e6b2b740 20 client.25106 trim_cache size 0 max 16384 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.282+0000 7f17e6b2b740 1 Processor -- start 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.282+0000 7f17e6b2b740 1 -- start start 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.282+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e6c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.282+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e65210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.282+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e69170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.282+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1acc90 con 0x55c462e62760 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.282+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ae7d0 con 0x55c462e63080 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.282+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ad360 con 0x55c462e6d940 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e69170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e69170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:42258/0 (socket says 172.21.15.19:42258) 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 -- 172.21.15.19:0/1293509279 learned_addr learned my addr 172.21.15.19:0/1293509279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e1ced640 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e65210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e24ee640 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e6c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 -- 172.21.15.19:0/1293509279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 msgr2=0x55c462e65210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e65210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 -- 172.21.15.19:0/1293509279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 msgr2=0x55c462e6c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e6c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 -- 172.21.15.19:0/1293509279 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c462e5a260 con 0x55c462e6d940 2024-09-19T04:07:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e24ee640 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e6c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.283+0000 7f17e2cef640 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e69170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f17d402aec0 tx=0x7f17d4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b15e0ef667e4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.284+0000 7f17c7fff640 1 -- 172.21.15.19:0/1293509279 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4031420 con 0x55c462e6d940 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.284+0000 7f17c7fff640 1 -- 172.21.15.19:0/1293509279 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d404b070 con 0x55c462e6d940 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.284+0000 7f17c7fff640 1 -- 172.21.15.19:0/1293509279 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d40044c0 con 0x55c462e6d940 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1293509279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 msgr2=0x55c462e69170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e69170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f17d402aec0 tx=0x7f17d4005ef0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1293509279 shutdown_connections 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e6d940 0x55c462e69170 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e63080 0x55c462e65210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/1293509279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e62760 0x55c462e6c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1293509279 >> 172.21.15.19:0/1293509279 conn(0x55c462e5ecf0 msgr2=0x55c462e5de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1293509279 shutdown_connections 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 -- 172.21.15.19:0/1293509279 wait complete. 2024-09-19T04:07:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 -- start start 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.285+0000 7f17e6b2b740 1 Processor -- start 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e75bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e63080 0x55c462e76130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e6b2b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e6d940 0x55c462e72850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ae100 con 0x55c462e63080 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ada30 con 0x55c462e6d940 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e6b2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1aac70 con 0x55c462e62760 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e6b2b740 10 client.0 _ll_register_callbacks cb 0x7f17cc00fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e1ced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e63080 0x55c462e76130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e24ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e75bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e1ced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e63080 0x55c462e76130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43782/0 (socket says 172.21.15.19:43782) 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e1ced640 1 -- 172.21.15.19:0/2168635265 learned_addr learned my addr 172.21.15.19:0/2168635265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.286+0000 7f17e2cef640 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e6d940 0x55c462e72850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e1ced640 1 -- 172.21.15.19:0/2168635265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 msgr2=0x55c462e75bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e1ced640 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e75bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e1ced640 1 -- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e6d940 msgr2=0x55c462e72850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e1ced640 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e6d940 0x55c462e72850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e1ced640 1 -- 172.21.15.19:0/2168635265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c462e72e40 con 0x55c462e63080 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e2cef640 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e6d940 0x55c462e72850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e1ced640 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e63080 0x55c462e76130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f17cc01e840 tx=0x7f17cc01e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8cad4f075cf994 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e24ee640 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e75bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17c5ffb640 1 -- 172.21.15.19:0/2168635265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc01fbd0 con 0x55c462e63080 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17c5ffb640 1 -- 172.21.15.19:0/2168635265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f17cc028470 con 0x55c462e63080 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.287+0000 7f17e6b2b740 1 -- 172.21.15.19:0/2168635265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d404b070 con 0x55c462e63080 2024-09-19T04:07:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17c5ffb640 1 -- 172.21.15.19:0/2168635265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17cc00fe70 con 0x55c462e63080 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17e6b2b740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17e6b2b740 1 -- 172.21.15.19:0/2168635265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c462ef1a20 con 0x55c462e63080 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17c5ffb640 1 -- 172.21.15.19:0/2168635265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d404b070 con 0x55c462e63080 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17c5ffb640 1 -- 172.21.15.19:0/2168635265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f17cc03c020 con 0x55c462e63080 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17e6b2b740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17e6b2b740 10 client.15495 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17e6b2b740 20 client.15495 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c462ef2240 0x55c462ef4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17e6b2b740 4 client.15495 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.288+0000 7f17e6b2b740 1 -- 172.21.15.19:0/2168635265 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x55c462dcfd20 con 0x55c462ef2240 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.289+0000 7f17e24ee640 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c462ef2240 0x55c462ef4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.289+0000 7f17c5ffb640 10 client.15495 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.289+0000 7f17e24ee640 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c462ef2240 0x55c462ef4660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e6f96fa669d980 server_cookie=e907fff3744d580e in_seq=0 out_seq=0 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.300+0000 7f17c5ffb640 1 -- 172.21.15.19:0/2168635265 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c462dcfd20 con 0x55c462ef2240 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.300+0000 7f17c5ffb640 10 client.15495 handle_command_reply: tid=0 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.300+0000 7f17e6b2b740 1 client.15495 shutdown 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.300+0000 7f17e6b2b740 1 -- 172.21.15.19:0/2168635265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e63080 msgr2=0x55c462e76130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.300+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e63080 0x55c462e76130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f17cc01e840 tx=0x7f17cc01e6e0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 1 -- 172.21.15.19:0/2168635265 shutdown_connections 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c462e6d940 0x55c462e72850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c462e63080 0x55c462e76130 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c462ef2240 0x55c462ef4660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17c5ffb640 0 client.15495 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 1 --2- 172.21.15.19:0/2168635265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c462e62760 0x55c462e75bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 1 -- 172.21.15.19:0/2168635265 >> 172.21.15.19:0/2168635265 conn(0x55c462e5ecf0 msgr2=0x55c462e5d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 1 -- 172.21.15.19:0/2168635265 shutdown_connections 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 1 -- 172.21.15.19:0/2168635265 wait complete. 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.301+0000 7f17e6b2b740 20 client.15495 trim_cache size 0 max 16384 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 29, 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24476, 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 2, 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "24fe3640": { 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 2, 2024-09-19T04:07:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 131.2, 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:07:47.209 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 -- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f17c80a32d0 msgr2=0x7f17c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f17c80a32d0 0x7f17c80a5790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005fd0 tx=0x7f17cc00b5d0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 -- 172.21.15.19:0/2467454351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17dc102dd0 msgr2=0x7f17dc198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17dc102dd0 0x7f17dc198580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f17d800d950 tx=0x7f17d800de10 comp rx=0 tx=0).stop 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 -- 172.21.15.19:0/2467454351 shutdown_connections 2024-09-19T04:07:47.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f17c80a32d0 0x7f17c80a5790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17dc108300 0x7f17dc19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc103780 0x7f17dc198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 --2- 172.21.15.19:0/2467454351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17dc102dd0 0x7f17dc198580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 -- 172.21.15.19:0/2467454351 >> 172.21.15.19:0/2467454351 conn(0x7f17dc0fc7d0 msgr2=0x7f17dc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 -- 172.21.15.19:0/2467454351 shutdown_connections 2024-09-19T04:07:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:46.302+0000 7f17e4752640 1 -- 172.21.15.19:0/2467454351 wait complete. 2024-09-19T04:07:48.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:48 smithi019 ceph-mon[30938]: pgmap v386: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:07:48.334 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '24fe3640', quiesce took 7.5 seconds. Will release after: 3.6999999999999993 2024-09-19T04:07:48.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:48 smithi111 ceph-mon[35365]: pgmap v386: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:07:48.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:48 smithi196 ceph-mon[35731]: pgmap v386: 129 pgs: 129 active+clean; 667 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T04:07:50.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:50 smithi111 ceph-mon[35365]: pgmap v387: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:07:50.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:50 smithi019 ceph-mon[30938]: pgmap v387: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:07:50.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:50 smithi196 ceph-mon[35731]: pgmap v387: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:07:52.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:07:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:52 smithi111 ceph-mon[35365]: pgmap v388: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T04:07:52.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:52 smithi019 ceph-mon[30938]: pgmap v388: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T04:07:52.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:52 smithi196 ceph-mon[35731]: pgmap v388: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T04:07:52.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2685469486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e8106f10 msgr2=0x7f50e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:52.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2685469486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e8106f10 0x7f50e810ea90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009ca0 tx=0x7f50e002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2685469486 shutdown_connections 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2685469486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e8106f10 0x7f50e810ea90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2685469486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50e8102790 0x7f50e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2685469486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8101de0 0x7f50e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2685469486 >> 172.21.15.19:0/2685469486 conn(0x7f50e80fd880 msgr2=0x7f50e80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2685469486 shutdown_connections 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.927+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2685469486 wait complete. 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.928+0000 7f50ee0fa640 1 Processor -- start 2024-09-19T04:07:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.928+0000 7f50ee0fa640 1 -- start start 2024-09-19T04:07:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50ee0fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e8101de0 0x7f50e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50ee0fa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50e8102790 0x7f50e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50ee0fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8106f10 0x7f50e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50ee0fa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50e8111fc0 con 0x7f50e8101de0 2024-09-19T04:07:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50ee0fa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f50e8111e40 con 0x7f50e8102790 2024-09-19T04:07:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50ee0fa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50e8112140 con 0x7f50e8106f10 2024-09-19T04:07:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e8101de0 0x7f50e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50e8102790 0x7f50e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50e8102790 0x7f50e8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:60476/0 (socket says 172.21.15.19:60476) 2024-09-19T04:07:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50e6ffd640 1 -- 172.21.15.19:0/2033976919 learned_addr learned my addr 172.21.15.19:0/2033976919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.929+0000 7f50e7fff640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8106f10 0x7f50e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e6ffd640 1 -- 172.21.15.19:0/2033976919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8106f10 msgr2=0x7f50e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e6ffd640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8106f10 0x7f50e8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e6ffd640 1 -- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e8101de0 msgr2=0x7f50e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e6ffd640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e8101de0 0x7f50e80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e6ffd640 1 -- 172.21.15.19:0/2033976919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e81a6860 con 0x7f50e8102790 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e6ffd640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50e8102790 0x7f50e8074a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00d950 tx=0x7f50dc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e4ff9640 1 -- 172.21.15.19:0/2033976919 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc017070 con 0x7f50e8102790 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e81a6b50 con 0x7f50e8102790 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e4ff9640 1 -- 172.21.15.19:0/2033976919 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50dc009c30 con 0x7f50e8102790 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50e4ff9640 1 -- 172.21.15.19:0/2033976919 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc00f6b0 con 0x7f50e8102790 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.930+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e81a70e0 con 0x7f50e8102790 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.932+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e81021c0 con 0x7f50e8102790 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.934+0000 7f50e4ff9640 1 -- 172.21.15.19:0/2033976919 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f50dc0109d0 con 0x7f50e8102790 2024-09-19T04:07:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.934+0000 7f50e4ff9640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50d00a32d0 0x7f50d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.935+0000 7f50e77fe640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50d00a32d0 0x7f50d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.935+0000 7f50e4ff9640 1 -- 172.21.15.19:0/2033976919 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f50dc0cf280 con 0x7f50e8102790 2024-09-19T04:07:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.935+0000 7f50e77fe640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50d00a32d0 0x7f50d00a5790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f50d8005d00 tx=0x7f50d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:52.939+0000 7f50e4ff9640 1 -- 172.21.15.19:0/2033976919 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f50dc009de0 con 0x7f50e8102790 2024-09-19T04:07:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.175+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50e81a72b0 con 0x7f50e8102790 2024-09-19T04:07:53.179 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:07:53.179 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:07:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.177+0000 7f50e4ff9640 1 -- 172.21.15.19:0/2033976919 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7f50dc09b450 con 0x7f50e8102790 2024-09-19T04:07:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:07:53.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.179+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50d00a32d0 msgr2=0x7f50d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:53.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.179+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50d00a32d0 0x7f50d00a5790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f50d8005d00 tx=0x7f50d800b410 comp rx=0 tx=0).stop 2024-09-19T04:07:53.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.180+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50e8102790 msgr2=0x7f50e8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:53.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.180+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50e8102790 0x7f50e8074a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00d950 tx=0x7f50dc00de10 comp rx=0 tx=0).stop 2024-09-19T04:07:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.183+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 shutdown_connections 2024-09-19T04:07:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.183+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50d00a32d0 0x7f50d00a5790 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.183+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8106f10 0x7f50e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.183+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50e8102790 0x7f50e8074a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.183+0000 7f50ee0fa640 1 --2- 172.21.15.19:0/2033976919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e8101de0 0x7f50e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.183+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 >> 172.21.15.19:0/2033976919 conn(0x7f50e80fd880 msgr2=0x7f50e80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.184+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 shutdown_connections 2024-09-19T04:07:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:53.184+0000 7f50ee0fa640 1 -- 172.21.15.19:0/2033976919 wait complete. 2024-09-19T04:07:54.015 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 24fe3640 --release --await' 2024-09-19T04:07:54.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 24fe3640 --release --await 2024-09-19T04:07:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:07:54.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:54 smithi111 ceph-mon[35365]: pgmap v389: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T04:07:54.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:54 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2033976919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:54.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:54 smithi019 ceph-mon[30938]: pgmap v389: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T04:07:54.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:54 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2033976919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:54.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:54 smithi196 ceph-mon[35731]: pgmap v389: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T04:07:54.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:54 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2033976919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:07:54.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/819395831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b081081f0 msgr2=0x7f7b08108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/819395831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b081081f0 0x7f7b08108610 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009930 tx=0x7f7afc02a420 comp rx=0 tx=0).stop 2024-09-19T04:07:54.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/819395831 shutdown_connections 2024-09-19T04:07:54.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/819395831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b08108b50 0x7f7b08115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/819395831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b081081f0 0x7f7b08108610 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/819395831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0810e230 0x7f7b0810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/819395831 >> 172.21.15.19:0/819395831 conn(0x7f7b08077bb0 msgr2=0x7f7b08079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:54.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/819395831 shutdown_connections 2024-09-19T04:07:54.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.862+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/819395831 wait complete. 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.863+0000 7f7b0e4e6640 1 Processor -- start 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.863+0000 7f7b0e4e6640 1 -- start start 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b0e4e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b081081f0 0x7f7b081a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b0e4e6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08108b50 0x7f7b081a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b0e4e6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0810e230 0x7f7b0819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b0e4e6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b08118550 con 0x7f7b08108b50 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08108b50 0x7f7b081a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b0e4e6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081183d0 con 0x7f7b0810e230 2024-09-19T04:07:54.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08108b50 0x7f7b081a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58912/0 (socket says 172.21.15.19:58912) 2024-09-19T04:07:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b0e4e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081186d0 con 0x7f7b081081f0 2024-09-19T04:07:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.864+0000 7f7b077fe640 1 -- 172.21.15.19:0/712897513 learned_addr learned my addr 172.21.15.19:0/712897513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b07fff640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b081081f0 0x7f7b081a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0810e230 0x7f7b0819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b077fe640 1 -- 172.21.15.19:0/712897513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b081081f0 msgr2=0x7f7b081a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b077fe640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b081081f0 0x7f7b081a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b077fe640 1 -- 172.21.15.19:0/712897513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0810e230 msgr2=0x7f7b0819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b077fe640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0810e230 0x7f7b0819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b077fe640 1 -- 172.21.15.19:0/712897513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0819efe0 con 0x7f7b08108b50 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0810e230 0x7f7b0819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b07fff640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b081081f0 0x7f7b081a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.865+0000 7f7b077fe640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08108b50 0x7f7b081a4c30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009230 tx=0x7f7afc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.866+0000 7f7b057fa640 1 -- 172.21.15.19:0/712897513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc03f030 con 0x7f7b08108b50 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.866+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/712897513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0819f270 con 0x7f7b08108b50 2024-09-19T04:07:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.866+0000 7f7b057fa640 1 -- 172.21.15.19:0/712897513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7afc004440 con 0x7f7b08108b50 2024-09-19T04:07:54.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.866+0000 7f7b057fa640 1 -- 172.21.15.19:0/712897513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc035070 con 0x7f7b08108b50 2024-09-19T04:07:54.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.866+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/712897513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b081ab5c0 con 0x7f7b08108b50 2024-09-19T04:07:54.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.868+0000 7f7b057fa640 1 -- 172.21.15.19:0/712897513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f7afc042030 con 0x7f7b08108b50 2024-09-19T04:07:54.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.869+0000 7f7b057fa640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7af00a3350 0x7f7af00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.869+0000 7f7b057fa640 1 -- 172.21.15.19:0/712897513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f7afc0efa50 con 0x7f7b08108b50 2024-09-19T04:07:54.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b108bf740 1 Processor -- start 2024-09-19T04:07:54.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b07fff640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7af00a3350 0x7f7af00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b108bf740 1 -- start start 2024-09-19T04:07:54.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919060 0x5573c2919440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919980 0x5573c2923d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b07fff640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7af00a3350 0x7f7af00a5810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b0819f9f0 tx=0x7f7af8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:54.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2926630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b108bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5573c287e620 con 0x5573c2919060 2024-09-19T04:07:54.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b108bf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5573c2825e50 con 0x5573c2924240 2024-09-19T04:07:54.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.870+0000 7f7b108bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5573c287a5e0 con 0x5573c2919980 2024-09-19T04:07:54.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919060 0x5573c2919440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919980 0x5573c2923d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919980 0x5573c2923d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39500/0 (socket says 172.21.15.19:39500) 2024-09-19T04:07:54.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919060 0x5573c2919440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58916/0 (socket says 172.21.15.19:58916) 2024-09-19T04:07:54.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b077fe640 1 -- 172.21.15.19:0/3139954324 learned_addr learned my addr 172.21.15.19:0/3139954324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:54.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2926630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b0ca83640 1 -- 172.21.15.19:0/3139954324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919980 msgr2=0x5573c2923d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919980 0x5573c2923d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b0ca83640 1 -- 172.21.15.19:0/3139954324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919060 msgr2=0x5573c2919440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919060 0x5573c2919440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b0ca83640 1 -- 172.21.15.19:0/3139954324 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c2816d40 con 0x5573c2924240 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b077fe640 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919980 0x5573c2923d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b07fff640 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919060 0x5573c2919440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.871+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2926630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7b00011760 tx=0x7f7b00011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbaa5233874f7ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7aee7fc640 1 -- 172.21.15.19:0/3139954324 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004d20 con 0x5573c2924240 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7aee7fc640 1 -- 172.21.15.19:0/3139954324 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0000f940 con 0x5573c2924240 2024-09-19T04:07:54.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7aee7fc640 1 -- 172.21.15.19:0/3139954324 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000044a0 con 0x5573c2924240 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7b108bf740 1 -- 172.21.15.19:0/3139954324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 msgr2=0x5573c2926630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2926630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7b00011760 tx=0x7f7b00011c20 comp rx=0 tx=0).stop 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7b108bf740 1 -- 172.21.15.19:0/3139954324 shutdown_connections 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2926630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919980 0x5573c2923d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3139954324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919060 0x5573c2919440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.872+0000 7f7b108bf740 1 -- 172.21.15.19:0/3139954324 >> 172.21.15.19:0/3139954324 conn(0x5573c29155f0 msgr2=0x5573c29159d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.873+0000 7f7b108bf740 1 -- 172.21.15.19:0/3139954324 shutdown_connections 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.873+0000 7f7b108bf740 1 -- 172.21.15.19:0/3139954324 wait complete. 2024-09-19T04:07:54.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.873+0000 7f7b108bf740 1 -- start start 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.873+0000 7f7b108bf740 1 Processor -- start 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919060 0x5573c29210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c2921630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2921b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b108bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5573c2926e10 con 0x5573c2919980 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b108bf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5573c2806f10 con 0x5573c2924240 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b108bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5573c2926f90 con 0x5573c2919060 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b108bf740 10 client.0 _ll_register_callbacks cb 0x5573c282adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b0ca83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2921b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b0ca83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2921b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:60502/0 (socket says 172.21.15.19:60502) 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b0ca83640 1 -- 172.21.15.19:0/3428337143 learned_addr learned my addr 172.21.15.19:0/3428337143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b077fe640 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c2921630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b0ca83640 1 -- 172.21.15.19:0/3428337143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919060 msgr2=0x5573c29210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:54.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919060 0x5573c29210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b0ca83640 1 -- 172.21.15.19:0/3428337143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 msgr2=0x5573c2921630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:54.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c2921630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b0ca83640 1 -- 172.21.15.19:0/3428337143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c258d580 con 0x5573c2924240 2024-09-19T04:07:54.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.874+0000 7f7b077fe640 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c2921630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:54.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.875+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2921b70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000fda0 tx=0x7f7b0000fdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eff833556020811 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:54.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.875+0000 7f7ae77fe640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-19T04:07:54.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.875+0000 7f7ae77fe640 1 -- 172.21.15.19:0/3428337143 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0002a060 con 0x5573c2924240 2024-09-19T04:07:54.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.875+0000 7f7ae77fe640 1 -- 172.21.15.19:0/3428337143 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f7b00005000 con 0x5573c2924240 2024-09-19T04:07:54.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.875+0000 7f7b108bf740 1 -- 172.21.15.19:0/3428337143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0000f940 con 0x5573c2924240 2024-09-19T04:07:54.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.875+0000 7f7ae77fe640 1 -- 172.21.15.19:0/3428337143 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b00031050 con 0x5573c2924240 2024-09-19T04:07:54.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.876+0000 7f7b108bf740 10 client.25130 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:54.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.876+0000 7f7b108bf740 1 -- 172.21.15.19:0/3428337143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573c2657960 con 0x5573c2924240 2024-09-19T04:07:54.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.876+0000 7f7ae77fe640 1 -- 172.21.15.19:0/3428337143 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0000f940 con 0x5573c2924240 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.877+0000 7f7ae77fe640 1 -- 172.21.15.19:0/3428337143 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7b00018070 con 0x5573c2924240 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.877+0000 7f7b108bf740 10 client.25130 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.877+0000 7f7b108bf740 10 client.25130 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.877+0000 7f7b108bf740 20 client.25130 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.877+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5573c2933d80 0x5573c2936170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.877+0000 7f7b108bf740 4 client.25130 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.877+0000 7f7b108bf740 1 -- 172.21.15.19:0/3428337143 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573c288bb90 con 0x5573c2933d80 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.877+0000 7f7b07fff640 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5573c2933d80 0x5573c2936170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.878+0000 7f7ae77fe640 10 client.25130 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.878+0000 7f7b07fff640 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5573c2933d80 0x5573c2936170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e558b58a308dd547 server_cookie=72d6aa278b2123f5 in_seq=0 out_seq=0 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.878+0000 7f7ae77fe640 1 -- 172.21.15.19:0/3428337143 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573c288bb90 con 0x5573c2933d80 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.878+0000 7f7ae77fe640 10 client.25130 handle_command_reply: tid=0 2024-09-19T04:07:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.878+0000 7f7b108bf740 1 client.25130 shutdown 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 -- 172.21.15.19:0/3428337143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 msgr2=0x5573c2921b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2921b70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000fda0 tx=0x7f7b0000fdd0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 -- 172.21.15.19:0/3428337143 shutdown_connections 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2924240 0x5573c2921b70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c2921630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5573c2933d80 0x5573c2936170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 --2- 172.21.15.19:0/3428337143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2919060 0x5573c29210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 -- 172.21.15.19:0/3428337143 >> 172.21.15.19:0/3428337143 conn(0x5573c29155f0 msgr2=0x5573c291b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7ae77fe640 0 client.25130 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 -- 172.21.15.19:0/3428337143 shutdown_connections 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 1 -- 172.21.15.19:0/3428337143 wait complete. 2024-09-19T04:07:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.879+0000 7f7b108bf740 20 client.25130 trim_cache size 0 max 16384 2024-09-19T04:07:54.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.913+0000 7f7b108bf740 1 Processor -- start 2024-09-19T04:07:54.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.913+0000 7f7b108bf740 1 -- start start 2024-09-19T04:07:54.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.913+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c2914640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c293b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 0x5573c293bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b108bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b5370 con 0x5573c2919980 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c293b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c293b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58930/0 (socket says 172.21.15.19:58930) 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b077fe640 1 -- 172.21.15.19:0/4284954080 learned_addr learned my addr 172.21.15.19:0/4284954080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b07fff640 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c2914640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 0x5573c293bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b108bf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b6eb0 con 0x5573c2919060 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b108bf740 1 -- 172.21.15.19:0/4284954080 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b5a40 con 0x5573c2924240 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b077fe640 1 -- 172.21.15.19:0/4284954080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 msgr2=0x5573c293bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:54.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b077fe640 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 0x5573c293bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b077fe640 1 -- 172.21.15.19:0/4284954080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 msgr2=0x5573c2914640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:54.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b077fe640 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c2914640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.914+0000 7f7b077fe640 1 -- 172.21.15.19:0/4284954080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c2915420 con 0x5573c2919980 2024-09-19T04:07:54.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.915+0000 7f7b077fe640 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c293b8b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0b8030 tx=0x7f7afc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc70d608362ef99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:54.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.915+0000 7f7aedffb640 1 -- 172.21.15.19:0/4284954080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc049040 con 0x5573c2919980 2024-09-19T04:07:54.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.915+0000 7f7aedffb640 1 -- 172.21.15.19:0/4284954080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7afc0c47e0 con 0x5573c2919980 2024-09-19T04:07:54.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.915+0000 7f7aedffb640 1 -- 172.21.15.19:0/4284954080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0bb070 con 0x5573c2919980 2024-09-19T04:07:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.916+0000 7f7b108bf740 1 -- 172.21.15.19:0/4284954080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 msgr2=0x5573c293b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.916+0000 7f7b108bf740 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c293b8b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0b8030 tx=0x7f7afc009260 comp rx=0 tx=0).stop 2024-09-19T04:07:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.916+0000 7f7b108bf740 1 -- 172.21.15.19:0/4284954080 shutdown_connections 2024-09-19T04:07:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.916+0000 7f7b108bf740 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 0x5573c293bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.916+0000 7f7b108bf740 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c293b8b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.916+0000 7f7b108bf740 1 --2- 172.21.15.19:0/4284954080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c2914640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.916+0000 7f7b108bf740 1 -- 172.21.15.19:0/4284954080 >> 172.21.15.19:0/4284954080 conn(0x5573c29155f0 msgr2=0x5573c2913f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.917+0000 7f7b108bf740 1 -- 172.21.15.19:0/4284954080 shutdown_connections 2024-09-19T04:07:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.917+0000 7f7b108bf740 1 -- 172.21.15.19:0/4284954080 wait complete. 2024-09-19T04:07:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.917+0000 7f7b108bf740 1 -- start start 2024-09-19T04:07:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.917+0000 7f7b108bf740 1 Processor -- start 2024-09-19T04:07:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c29282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b108bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c29287e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c29282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c29282a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:60522/0 (socket says 172.21.15.19:60522) 2024-09-19T04:07:54.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c29287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c29287e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58938/0 (socket says 172.21.15.19:58938) 2024-09-19T04:07:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b07fff640 1 -- 172.21.15.19:0/1879568647 learned_addr learned my addr 172.21.15.19:0/1879568647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b108bf740 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 0x5573c299d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b67e0 con 0x5573c2919980 2024-09-19T04:07:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b6110 con 0x5573c2919060 2024-09-19T04:07:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.918+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b3350 con 0x5573c2924240 2024-09-19T04:07:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.919+0000 7f7b0ca83640 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 0x5573c299d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.919+0000 7f7b077fe640 1 -- 172.21.15.19:0/1879568647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 msgr2=0x5573c299d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.919+0000 7f7b077fe640 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 0x5573c299d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.919+0000 7f7b077fe640 1 -- 172.21.15.19:0/1879568647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 msgr2=0x5573c29282a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.919+0000 7f7b077fe640 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c29282a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.919+0000 7f7b077fe640 1 -- 172.21.15.19:0/1879568647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c29290a0 con 0x5573c2919980 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.919+0000 7f7b077fe640 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c29287e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5573c282adf0 tx=0x7f7afc0b7ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c65e3deb92bc0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7ae77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7ae77fe640 1 -- 172.21.15.19:0/1879568647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0d8070 con 0x5573c2919980 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7ae77fe640 1 -- 172.21.15.19:0/1879568647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f7afc031dd0 con 0x5573c2919980 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7ae77fe640 10 client.0 inactive, discarding osd_map(93..93 src has 1..93) 2024-09-19T04:07:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7b108bf740 10 client.0 _ll_register_callbacks cb 0x7f7b00011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc0c47e0 con 0x5573c2919980 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7ae77fe640 1 -- 172.21.15.19:0/1879568647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7afc09f050 con 0x5573c2919980 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7b108bf740 10 client.15519 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.920+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5573c29afb50 con 0x5573c2919980 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7ae77fe640 1 -- 172.21.15.19:0/1879568647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7afc0da2f0 con 0x5573c2919980 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7b108bf740 10 client.15519 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7b108bf740 10 client.15519 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7b108bf740 20 client.15519 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7b108bf740 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5573c29b00e0 0x5573c29b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7b108bf740 4 client.15519 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "24fe3640", "release": true, "await": true}] 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "quiesce db", "set_id": "24fe3640", "release": true, "await": true}) -- 0x7f7afc0c47e0 con 0x5573c29b00e0 2024-09-19T04:07:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7b07fff640 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5573c29b00e0 0x5573c29b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7b07fff640 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5573c29b00e0 0x5573c29b25a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890e7353833368c4 server_cookie=cd163f2e2ac63027 in_seq=0 out_seq=0 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.921+0000 7f7ae77fe640 10 client.15519 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.923+0000 7f7ae77fe640 1 -- 172.21.15.19:0/1879568647 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f7afc0c47e0 con 0x5573c29b00e0 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.923+0000 7f7ae77fe640 10 client.15519 handle_command_reply: tid=0 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 client.15519 shutdown 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 msgr2=0x5573c29287e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c29287e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5573c282adf0 tx=0x7f7afc0b7ec0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 shutdown_connections 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5573c29b00e0 0x5573c29b25a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7ae77fe640 0 client.15519 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c2924240 0x5573c299d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573c2919980 0x5573c29287e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 --2- 172.21.15.19:0/1879568647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573c2919060 0x5573c29282a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.924+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 >> 172.21.15.19:0/1879568647 conn(0x5573c29155f0 msgr2=0x5573c291b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.926+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 shutdown_connections 2024-09-19T04:07:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.926+0000 7f7b108bf740 1 -- 172.21.15.19:0/1879568647 wait complete. 2024-09-19T04:07:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.926+0000 7f7b108bf740 20 client.15519 trim_cache size 0 max 16384 2024-09-19T04:07:54.931 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:07:54.931 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 29, 2024-09-19T04:07:54.931 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24476, 2024-09-19T04:07:54.931 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 4, 2024-09-19T04:07:54.931 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "24fe3640": { 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 4, 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 131.2, 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-19T04:07:54.932 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:07:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7af00a3350 msgr2=0x7f7af00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7af00a3350 0x7f7af00a5810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b0819f9f0 tx=0x7f7af8005f90 comp rx=0 tx=0).stop 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08108b50 msgr2=0x7f7b081a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08108b50 0x7f7b081a4c30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009230 tx=0x7f7afc004040 comp rx=0 tx=0).stop 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/712897513 shutdown_connections 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7af00a3350 0x7f7af00a5810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0810e230 0x7f7b0819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08108b50 0x7f7b081a4c30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 --2- 172.21.15.19:0/712897513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b081081f0 0x7f7b081a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.927+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/712897513 >> 172.21.15.19:0/712897513 conn(0x7f7b08077bb0 msgr2=0x7f7b08079510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.928+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/712897513 shutdown_connections 2024-09-19T04:07:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:07:54.928+0000 7f7b0e4e6640 1 -- 172.21.15.19:0/712897513 wait complete. 2024-09-19T04:07:55.862 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '24fe3640', total seconds elapsed: 15.0 2024-09-19T04:07:56.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:56 smithi019 ceph-mon[30938]: pgmap v390: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:07:56.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:56 smithi111 ceph-mon[35365]: pgmap v390: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:07:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:56 smithi196 ceph-mon[35731]: pgmap v390: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.973+0000 7f8159ad0640 1 -- 172.21.15.19:0/1886064454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81541025c0 msgr2=0x7f81541049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.973+0000 7f8159ad0640 1 --2- 172.21.15.19:0/1886064454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81541025c0 0x7f81541049b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8144009900 tx=0x7f814402a410 comp rx=0 tx=0).stop 2024-09-19T04:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.973+0000 7f8159ad0640 1 -- 172.21.15.19:0/1886064454 shutdown_connections 2024-09-19T04:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.973+0000 7f8159ad0640 1 --2- 172.21.15.19:0/1886064454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8154107820 0x7f8154109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.973+0000 7f8159ad0640 1 --2- 172.21.15.19:0/1886064454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8154104ef0 0x7f81541072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.973+0000 7f8159ad0640 1 --2- 172.21.15.19:0/1886064454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81541025c0 0x7f81541049b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.973+0000 7f8159ad0640 1 -- 172.21.15.19:0/1886064454 >> 172.21.15.19:0/1886064454 conn(0x7f81540fc1e0 msgr2=0x7f81540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.974+0000 7f8159ad0640 1 -- 172.21.15.19:0/1886064454 shutdown_connections 2024-09-19T04:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.974+0000 7f8159ad0640 1 -- 172.21.15.19:0/1886064454 wait complete. 2024-09-19T04:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.975+0000 7f8159ad0640 1 Processor -- start 2024-09-19T04:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.975+0000 7f8159ad0640 1 -- start start 2024-09-19T04:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.975+0000 7f8159ad0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81541025c0 0x7f8154197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.975+0000 7f8159ad0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8154104ef0 0x7f81541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8159ad0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8154107820 0x7f815419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8159ad0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f815410d870 con 0x7f8154104ef0 2024-09-19T04:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8159ad0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f815410d6f0 con 0x7f8154107820 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8159ad0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f815410d9f0 con 0x7f81541025c0 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8153fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8154107820 0x7f815419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f81537fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81541025c0 0x7f8154197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8153fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8154107820 0x7f815419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:60550/0 (socket says 172.21.15.19:60550) 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8153fff640 1 -- 172.21.15.19:0/238777845 learned_addr learned my addr 172.21.15.19:0/238777845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8152ffd640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8154104ef0 0x7f81541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8153fff640 1 -- 172.21.15.19:0/238777845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81541025c0 msgr2=0x7f8154197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8153fff640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81541025c0 0x7f8154197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8153fff640 1 -- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8154104ef0 msgr2=0x7f81541983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8153fff640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8154104ef0 0x7f81541983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8153fff640 1 -- 172.21.15.19:0/238777845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815419fa90 con 0x7f8154107820 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f81537fe640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81541025c0 0x7f8154197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.976+0000 7f8152ffd640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8154104ef0 0x7f81541983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.977+0000 7f8153fff640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8154107820 0x7f815419f420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f814c00e9d0 tx=0x7f814c00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.977+0000 7f8150ff9640 1 -- 172.21.15.19:0/238777845 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c00cd80 con 0x7f8154107820 2024-09-19T04:07:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.977+0000 7f8150ff9640 1 -- 172.21.15.19:0/238777845 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c005020 con 0x7f8154107820 2024-09-19T04:07:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.978+0000 7f8150ff9640 1 -- 172.21.15.19:0/238777845 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c0116d0 con 0x7f8154107820 2024-09-19T04:07:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.978+0000 7f8159ad0640 1 -- 172.21.15.19:0/238777845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8154100410 con 0x7f8154107820 2024-09-19T04:07:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.978+0000 7f8159ad0640 1 -- 172.21.15.19:0/238777845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81541008b0 con 0x7f8154107820 2024-09-19T04:07:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.981+0000 7f815c86c740 1 Processor -- start 2024-09-19T04:07:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f8150ff9640 1 -- 172.21.15.19:0/238777845 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f814c0051d0 con 0x7f8154107820 2024-09-19T04:07:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f815c86c740 1 -- start start 2024-09-19T04:07:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f815c86c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e169a0 0x55af87e16da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f815c86c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e1bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f815c86c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e1e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f815c86c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af87d7a2f0 con 0x55af87e1c370 2024-09-19T04:07:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f815c86c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55af87d22270 con 0x55af87e169a0 2024-09-19T04:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f815c86c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af87d762b0 con 0x55af87e172e0 2024-09-19T04:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f81537fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e1bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f81537fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e1bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39572/0 (socket says 172.21.15.19:39572) 2024-09-19T04:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f81537fe640 1 -- 172.21.15.19:0/1855736653 learned_addr learned my addr 172.21.15.19:0/1855736653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.982+0000 7f8152ffd640 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e1e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f81537fe640 1 -- 172.21.15.19:0/1855736653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e169a0 msgr2=0x55af87e16da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f81537fe640 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e169a0 0x55af87e16da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f81537fe640 1 -- 172.21.15.19:0/1855736653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 msgr2=0x55af87e1e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f81537fe640 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e1e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f81537fe640 1 -- 172.21.15.19:0/1855736653 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af87d21340 con 0x55af87e172e0 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f8152ffd640 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e1e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f8150ff9640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f813c0a3390 0x7f813c0a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f81537fe640 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e1bda0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8144009900 tx=0x7f81440315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a218877b6894455 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.983+0000 7f8152ffd640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f813c0a3390 0x7f813c0a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.984+0000 7f8150ff9640 1 -- 172.21.15.19:0/238777845 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f814c0d0380 con 0x7f8154107820 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.984+0000 7f8139ffb640 1 -- 172.21.15.19:0/1855736653 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144035070 con 0x55af87e172e0 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.984+0000 7f8152ffd640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f813c0a3390 0x7f813c0a5850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8154199380 tx=0x7f8148005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.984+0000 7f8139ffb640 1 -- 172.21.15.19:0/1855736653 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8144004770 con 0x55af87e172e0 2024-09-19T04:07:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.984+0000 7f8139ffb640 1 -- 172.21.15.19:0/1855736653 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81440319d0 con 0x55af87e172e0 2024-09-19T04:07:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 -- 172.21.15.19:0/1855736653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 msgr2=0x55af87e1bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e1bda0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8144009900 tx=0x7f81440315b0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 -- 172.21.15.19:0/1855736653 shutdown_connections 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e1e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e1bda0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 --2- 172.21.15.19:0/1855736653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e169a0 0x55af87e16da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 -- 172.21.15.19:0/1855736653 >> 172.21.15.19:0/1855736653 conn(0x55af87e12010 msgr2=0x55af87e12410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 -- 172.21.15.19:0/1855736653 shutdown_connections 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.985+0000 7f815c86c740 1 -- 172.21.15.19:0/1855736653 wait complete. 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 1 -- start start 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 1 Processor -- start 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e169a0 0x55af87e29370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e172e0 0x55af87e298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e1c370 0x55af87e308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af87e1f2e0 con 0x55af87e169a0 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55af87d03290 con 0x55af87e172e0 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af87e1f460 con 0x55af87e1c370 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f815c86c740 10 client.0 _ll_register_callbacks cb 0x55af87d34140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.986+0000 7f81537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e169a0 0x55af87e29370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f81537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e169a0 0x55af87e29370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58968/0 (socket says 172.21.15.19:58968) 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f81537fe640 1 -- 172.21.15.19:0/713784069 learned_addr learned my addr 172.21.15.19:0/713784069 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f8152ffd640 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e172e0 0x55af87e298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f8153fff640 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e1c370 0x55af87e308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f81537fe640 1 -- 172.21.15.19:0/713784069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e1c370 msgr2=0x55af87e308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f81537fe640 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e1c370 0x55af87e308f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f81537fe640 1 -- 172.21.15.19:0/713784069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e172e0 msgr2=0x55af87e298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f81537fe640 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e172e0 0x55af87e298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f81537fe640 1 -- 172.21.15.19:0/713784069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af87a8d210 con 0x55af87e169a0 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f81537fe640 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e169a0 0x55af87e29370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8144031440 tx=0x7f8144005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e6a82764bc3182 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.987+0000 7f8132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.988+0000 7f8132ffd640 1 -- 172.21.15.19:0/713784069 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144035070 con 0x55af87e169a0 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.988+0000 7f815c86c740 1 -- 172.21.15.19:0/713784069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8144004770 con 0x55af87e169a0 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.988+0000 7f8132ffd640 1 -- 172.21.15.19:0/713784069 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f8144031790 con 0x55af87e169a0 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.988+0000 7f8132ffd640 1 -- 172.21.15.19:0/713784069 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f814405a050 con 0x55af87e169a0 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.988+0000 7f815c86c740 10 client.15531 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.988+0000 7f815c86c740 1 -- 172.21.15.19:0/713784069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af87b59620 con 0x55af87e169a0 2024-09-19T04:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.988+0000 7f8132ffd640 1 -- 172.21.15.19:0/713784069 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8144004770 con 0x55af87e169a0 2024-09-19T04:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.989+0000 7f8132ffd640 1 -- 172.21.15.19:0/713784069 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8144055030 con 0x55af87e169a0 2024-09-19T04:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.989+0000 7f815c86c740 10 client.15531 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.989+0000 7f815c86c740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.989+0000 7f815c86c740 20 client.15531 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.989+0000 7f815c86c740 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55af87e23bc0 0x55af87e26010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.989+0000 7f815c86c740 4 client.15531 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.989+0000 7f815c86c740 1 -- 172.21.15.19:0/713784069 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af87d87a20 con 0x55af87e23bc0 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.990+0000 7f8152ffd640 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55af87e23bc0 0x55af87e26010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.990+0000 7f8132ffd640 10 client.15531 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.990+0000 7f8152ffd640 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55af87e23bc0 0x55af87e26010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f8cd5048fd4f1a server_cookie=1e80070354993273 in_seq=0 out_seq=0 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f8132ffd640 1 -- 172.21.15.19:0/713784069 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af87d87a20 con 0x55af87e23bc0 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f8132ffd640 10 client.15531 handle_command_reply: tid=0 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 client.15531 shutdown 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 -- 172.21.15.19:0/713784069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e169a0 msgr2=0x55af87e29370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e169a0 0x55af87e29370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8144031440 tx=0x7f8144005210 comp rx=0 tx=0).stop 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 -- 172.21.15.19:0/713784069 shutdown_connections 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55af87e23bc0 0x55af87e26010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e1c370 0x55af87e308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e172e0 0x55af87e298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 --2- 172.21.15.19:0/713784069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e169a0 0x55af87e29370 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f8132ffd640 0 client.15531 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 -- 172.21.15.19:0/713784069 >> 172.21.15.19:0/713784069 conn(0x55af87e12010 msgr2=0x55af87e13820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 -- 172.21.15.19:0/713784069 shutdown_connections 2024-09-19T04:07:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 1 -- 172.21.15.19:0/713784069 wait complete. 2024-09-19T04:07:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:56.991+0000 7f815c86c740 20 client.15531 trim_cache size 0 max 16384 2024-09-19T04:07:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f815c86c740 1 Processor -- start 2024-09-19T04:07:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f815c86c740 1 -- start start 2024-09-19T04:07:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f815c86c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e172e0 0x55af87e29830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f815c86c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e1c370 0x55af87e30880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f815c86c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e12e90 0x55af87e13340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f815c86c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81541a8380 con 0x55af87e172e0 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f815c86c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81541a9ec0 con 0x55af87e1c370 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f815c86c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81541a8a50 con 0x55af87e12e90 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f81537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e172e0 0x55af87e29830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f81537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e172e0 0x55af87e29830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58976/0 (socket says 172.21.15.19:58976) 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.020+0000 7f81537fe640 1 -- 172.21.15.19:0/1982591660 learned_addr learned my addr 172.21.15.19:0/1982591660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f8152ffd640 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e1c370 0x55af87e30880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81537fe640 1 -- 172.21.15.19:0/1982591660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e12e90 msgr2=0x55af87e13340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81537fe640 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e12e90 0x55af87e13340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81537fe640 1 -- 172.21.15.19:0/1982591660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e1c370 msgr2=0x55af87e30880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81537fe640 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e1c370 0x55af87e30880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81537fe640 1 -- 172.21.15.19:0/1982591660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af87e0d2d0 con 0x55af87e172e0 2024-09-19T04:07:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f8152ffd640 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e1c370 0x55af87e30880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81537fe640 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e172e0 0x55af87e29830 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f814402a450 tx=0x7f8144005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a98c7347e31fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81397fa640 1 -- 172.21.15.19:0/1982591660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144035040 con 0x55af87e172e0 2024-09-19T04:07:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81397fa640 1 -- 172.21.15.19:0/1982591660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8144063070 con 0x55af87e172e0 2024-09-19T04:07:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.021+0000 7f81397fa640 1 -- 172.21.15.19:0/1982591660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814404d700 con 0x55af87e172e0 2024-09-19T04:07:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 -- 172.21.15.19:0/1982591660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e172e0 msgr2=0x55af87e29830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e172e0 0x55af87e29830 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f814402a450 tx=0x7f8144005fd0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 -- 172.21.15.19:0/1982591660 shutdown_connections 2024-09-19T04:07:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e12e90 0x55af87e13340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e1c370 0x55af87e30880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 --2- 172.21.15.19:0/1982591660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e172e0 0x55af87e29830 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 -- 172.21.15.19:0/1982591660 >> 172.21.15.19:0/1982591660 conn(0x55af87e12010 msgr2=0x55af87e29120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 -- 172.21.15.19:0/1982591660 shutdown_connections 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.022+0000 7f815c86c740 1 -- 172.21.15.19:0/1982591660 wait complete. 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.023+0000 7f815c86c740 1 -- start start 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.023+0000 7f815c86c740 1 Processor -- start 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.023+0000 7f815c86c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e12e90 0x55af87e2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.023+0000 7f815c86c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e2cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.023+0000 7f815c86c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e2faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f815c86c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81541a97f0 con 0x55af87e1c370 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f815c86c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81541a9120 con 0x55af87e12e90 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f815c86c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81541a6410 con 0x55af87e172e0 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e2faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e2faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58978/0 (socket says 172.21.15.19:58978) 2024-09-19T04:07:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 -- 172.21.15.19:0/1125872265 learned_addr learned my addr 172.21.15.19:0/1125872265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f815c86c740 10 client.0 _ll_register_callbacks cb 0x7f8148009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8152ffd640 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e2cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f81537fe640 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e12e90 0x55af87e2c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 -- 172.21.15.19:0/1125872265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 msgr2=0x55af87e2cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e2cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 -- 172.21.15.19:0/1125872265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e12e90 msgr2=0x55af87e2c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e12e90 0x55af87e2c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 -- 172.21.15.19:0/1125872265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af87e30100 con 0x55af87e1c370 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8152ffd640 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e2cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8153fff640 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e2faf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f814c09c250 tx=0x7f814c0a0980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea32202d590c58a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:07:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8132ffd640 1 -- 172.21.15.19:0/1125872265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c01f050 con 0x55af87e1c370 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f8132ffd640 1 -- 172.21.15.19:0/1125872265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f814c005540 con 0x55af87e1c370 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.024+0000 7f815c86c740 1 -- 172.21.15.19:0/1125872265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8144063070 con 0x55af87e1c370 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f8132ffd640 1 -- 172.21.15.19:0/1125872265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f814c0cdca0 con 0x55af87e1c370 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f815c86c740 10 client.15537 fetch_fsmap learned FSMap version 29 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f815c86c740 1 -- 172.21.15.19:0/1125872265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af87ea84f0 con 0x55af87e1c370 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f8132ffd640 1 -- 172.21.15.19:0/1125872265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c07c050 con 0x55af87e1c370 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f8132ffd640 1 -- 172.21.15.19:0/1125872265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f814c00f040 con 0x55af87e1c370 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f815c86c740 10 client.15537 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f815c86c740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f815c86c740 20 client.15537 populate_metadata read hostname 'smithi019' 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f815c86c740 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55af87ea8e20 0x55af87eab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f815c86c740 4 client.15537 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.025+0000 7f815c86c740 1 -- 172.21.15.19:0/1125872265 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8144063070 con 0x55af87ea8e20 2024-09-19T04:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.026+0000 7f81537fe640 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55af87ea8e20 0x55af87eab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.026+0000 7f8132ffd640 10 client.15537 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.026+0000 7f81537fe640 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55af87ea8e20 0x55af87eab240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06d2a15dc1a2e85 server_cookie=7a246bdd71bb4c0f in_seq=0 out_seq=0 2024-09-19T04:07:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.033+0000 7f8132ffd640 1 -- 172.21.15.19:0/1125872265 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8144063070 con 0x55af87ea8e20 2024-09-19T04:07:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f8132ffd640 10 client.15537 handle_command_reply: tid=0 2024-09-19T04:07:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 client.15537 shutdown 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 -- 172.21.15.19:0/1125872265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 msgr2=0x55af87e2faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e2faf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f814c09c250 tx=0x7f814c0a0980 comp rx=0 tx=0).stop 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 -- 172.21.15.19:0/1125872265 shutdown_connections 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af87e1c370 0x55af87e2faf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55af87ea8e20 0x55af87eab240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f8132ffd640 0 client.15537 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af87e172e0 0x55af87e2cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 --2- 172.21.15.19:0/1125872265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55af87e12e90 0x55af87e2c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 -- 172.21.15.19:0/1125872265 >> 172.21.15.19:0/1125872265 conn(0x55af87e12010 msgr2=0x55af87e132e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 -- 172.21.15.19:0/1125872265 shutdown_connections 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 1 -- 172.21.15.19:0/1125872265 wait complete. 2024-09-19T04:07:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.034+0000 7f815c86c740 20 client.15537 trim_cache size 0 max 16384 2024-09-19T04:07:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:07:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:07:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:07:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 -- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f813c0a3390 msgr2=0x7f813c0a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f813c0a3390 0x7f813c0a5850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8154199380 tx=0x7f8148005f90 comp rx=0 tx=0).stop 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 -- 172.21.15.19:0/238777845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8154107820 msgr2=0x7f815419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8154107820 0x7f815419f420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f814c00e9d0 tx=0x7f814c00ee90 comp rx=0 tx=0).stop 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 -- 172.21.15.19:0/238777845 shutdown_connections 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f813c0a3390 0x7f813c0a5850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8154107820 0x7f815419f420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8154104ef0 0x7f81541983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 --2- 172.21.15.19:0/238777845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81541025c0 0x7f8154197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:07:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 -- 172.21.15.19:0/238777845 >> 172.21.15.19:0/238777845 conn(0x7f81540fc1e0 msgr2=0x7f8154102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:07:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 -- 172.21.15.19:0/238777845 shutdown_connections 2024-09-19T04:07:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:07:57.035+0000 7f8159ad0640 1 -- 172.21.15.19:0/238777845 wait complete. 2024-09-19T04:07:58.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:07:58 smithi111 ceph-mon[35365]: pgmap v391: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:07:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:07:58 smithi019 ceph-mon[30938]: pgmap v391: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:07:58.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:07:58 smithi196 ceph-mon[35731]: pgmap v391: 129 pgs: 129 active+clean; 673 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:08:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:00 smithi111 ceph-mon[35365]: pgmap v392: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-19T04:08:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:00 smithi019 ceph-mon[30938]: pgmap v392: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-19T04:08:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:00 smithi196 ceph-mon[35731]: pgmap v392: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-19T04:08:02.408 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:02 smithi111 ceph-mon[35365]: pgmap v393: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:02.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:02 smithi019 ceph-mon[30938]: pgmap v393: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:02.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:02 smithi196 ceph-mon[35731]: pgmap v393: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:04.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:04 smithi019 ceph-mon[30938]: pgmap v394: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:04.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:04 smithi111 ceph-mon[35365]: pgmap v394: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:04 smithi196 ceph-mon[35731]: pgmap v394: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:06.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:06 smithi111 ceph-mon[35365]: pgmap v395: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T04:08:06.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:06 smithi019 ceph-mon[30938]: pgmap v395: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T04:08:06.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:06 smithi196 ceph-mon[35731]: pgmap v395: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T04:08:08.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:08 smithi111 ceph-mon[35365]: pgmap v396: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:08.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:08 smithi019 ceph-mon[30938]: pgmap v396: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:08 smithi196 ceph-mon[35731]: pgmap v396: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T04:08:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 -- 172.21.15.19:0/3540852429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44102d50 msgr2=0x7fdc441031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/3540852429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44102d50 0x7fdc441031d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009900 tx=0x7fdc3802a410 comp rx=0 tx=0).stop 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 -- 172.21.15.19:0/3540852429 shutdown_connections 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/3540852429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc44103710 0x7fdc44109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/3540852429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44102d50 0x7fdc441031d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/3540852429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc44101b60 0x7fdc44101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 -- 172.21.15.19:0/3540852429 >> 172.21.15.19:0/3540852429 conn(0x7fdc440fd2e0 msgr2=0x7fdc440ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 -- 172.21.15.19:0/3540852429 shutdown_connections 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.170+0000 7fdc48ec3640 1 -- 172.21.15.19:0/3540852429 wait complete. 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.171+0000 7fdc48ec3640 1 Processor -- start 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.171+0000 7fdc48ec3640 1 -- start start 2024-09-19T04:08:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.171+0000 7fdc48ec3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc44101b60 0x7fdc44075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.171+0000 7fdc48ec3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc44102d50 0x7fdc440745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.171+0000 7fdc48ec3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44103710 0x7fdc44074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc42d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44103710 0x7fdc44074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc42d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44103710 0x7fdc44074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48070/0 (socket says 172.21.15.19:48070) 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc42d9d640 1 -- 172.21.15.19:0/317725524 learned_addr learned my addr 172.21.15.19:0/317725524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4410d810 con 0x7fdc44103710 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4410d690 con 0x7fdc44101b60 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4410d990 con 0x7fdc44102d50 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc4259c640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc44101b60 0x7fdc44075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc44102d50 0x7fdc440745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc4259c640 1 -- 172.21.15.19:0/317725524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc44102d50 msgr2=0x7fdc440745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc4259c640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc44102d50 0x7fdc440745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc4259c640 1 -- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44103710 msgr2=0x7fdc44074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc4259c640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44103710 0x7fdc44074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc4259c640 1 -- 172.21.15.19:0/317725524 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc440753b0 con 0x7fdc44101b60 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc44102d50 0x7fdc440745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.172+0000 7fdc4259c640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc44101b60 0x7fdc44075fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400e9c0 tx=0x7fdc3400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.173+0000 7fdc337fe640 1 -- 172.21.15.19:0/317725524 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400ccc0 con 0x7fdc44101b60 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.173+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc441a1f00 con 0x7fdc44101b60 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.173+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc441a2360 con 0x7fdc44101b60 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.173+0000 7fdc337fe640 1 -- 172.21.15.19:0/317725524 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc340035f0 con 0x7fdc44101b60 2024-09-19T04:08:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.173+0000 7fdc337fe640 1 -- 172.21.15.19:0/317725524 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400ce60 con 0x7fdc44101b60 2024-09-19T04:08:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.175+0000 7fdc337fe640 1 -- 172.21.15.19:0/317725524 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fdc340037a0 con 0x7fdc44101b60 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.176+0000 7fdc337fe640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc2c0a3410 0x7fdc2c0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.176+0000 7fdc337fe640 1 -- 172.21.15.19:0/317725524 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fdc340d02a0 con 0x7fdc44101b60 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.176+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc2c0a3410 0x7fdc2c0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.176+0000 7fdc4bc5d740 1 Processor -- start 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc4bc5d740 1 -- start start 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc2c0a3410 0x7fdc2c0a58d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006010 tx=0x7fdc38000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c18fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c194da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c197830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc42d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc41d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c194da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc42d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48074/0 (socket says 172.21.15.19:48074) 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc41d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c194da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:41618/0 (socket says 172.21.15.19:41618) 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc42d9d640 1 -- 172.21.15.19:0/2556557307 learned_addr learned my addr 172.21.15.19:0/2556557307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/2556557307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c72c0f32f0 con 0x55c72c195370 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc4259c640 1 --2- 172.21.15.19:0/2556557307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c18fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/2556557307 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c72c09b270 con 0x55c72c18f9a0 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.177+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/2556557307 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c72c0ef2b0 con 0x55c72c1902e0 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc42d9d640 1 -- 172.21.15.19:0/2556557307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 msgr2=0x55c72c194da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/2556557307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c194da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc42d9d640 1 -- 172.21.15.19:0/2556557307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 msgr2=0x55c72c18fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/2556557307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c18fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc42d9d640 1 -- 172.21.15.19:0/2556557307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c72c09a340 con 0x55c72c195370 2024-09-19T04:08:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/2556557307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c197830 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0104d0 tx=0x7fdc3c010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4c911970646bca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc30ff9640 1 -- 172.21.15.19:0/2556557307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c011760 con 0x55c72c195370 2024-09-19T04:08:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc30ff9640 1 -- 172.21.15.19:0/2556557307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc3c011de0 con 0x55c72c195370 2024-09-19T04:08:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.178+0000 7fdc30ff9640 1 -- 172.21.15.19:0/2556557307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0193d0 con 0x55c72c195370 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.179+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/2556557307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 msgr2=0x55c72c197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.179+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/2556557307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c197830 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0104d0 tx=0x7fdc3c010990 comp rx=0 tx=0).stop 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.180+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/2556557307 shutdown_connections 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.180+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/2556557307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c197830 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.180+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/2556557307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c194da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.180+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/2556557307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c18fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.180+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/2556557307 >> 172.21.15.19:0/2556557307 conn(0x55c72c18b010 msgr2=0x55c72c18b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.180+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/2556557307 shutdown_connections 2024-09-19T04:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.180+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/2556557307 wait complete. 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.180+0000 7fdc4bc5d740 1 -- start start 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc4bc5d740 1 Processor -- start 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c195370 0x55c72c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c72c198340 con 0x55c72c1902e0 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc41d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c72c07c290 con 0x55c72c18f9a0 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c72c1984c0 con 0x55c72c195370 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc41d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48086/0 (socket says 172.21.15.19:48086) 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc41d9b640 1 -- 172.21.15.19:0/187533232 learned_addr learned my addr 172.21.15.19:0/187533232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.181+0000 7fdc4259c640 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c195370 0x55c72c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc4259c640 1 -- 172.21.15.19:0/187533232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c195370 msgr2=0x55c72c1a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc4259c640 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c195370 0x55c72c1a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc4259c640 1 -- 172.21.15.19:0/187533232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 msgr2=0x55c72c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc4259c640 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc4259c640 1 -- 172.21.15.19:0/187533232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c72be06210 con 0x55c72c18f9a0 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc4bc5d740 10 client.0 _ll_register_callbacks cb 0x55c72c0ad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc4259c640 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c199ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc340cd8b0 tx=0x7fdc340d0180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e99ab63931fb993b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc29ffb640 1 -- 172.21.15.19:0/187533232 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc340a5030 con 0x55c72c18f9a0 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/187533232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc3c011de0 con 0x55c72c18f9a0 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.182+0000 7fdc29ffb640 1 -- 172.21.15.19:0/187533232 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fdc340cdb00 con 0x55c72c18f9a0 2024-09-19T04:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.183+0000 7fdc29ffb640 1 -- 172.21.15.19:0/187533232 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc340b5260 con 0x55c72c18f9a0 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.183+0000 7fdc4bc5d740 10 client.25163 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.183+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/187533232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55c72bed2620 con 0x55c72c18f9a0 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.183+0000 7fdc29ffb640 1 -- 172.21.15.19:0/187533232 <== mon.2 v2:172.21.15.111:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdc340a4ba0 con 0x55c72c18f9a0 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.183+0000 7fdc4bc5d740 10 client.25163 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.183+0000 7fdc4bc5d740 10 client.25163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.183+0000 7fdc4bc5d740 20 client.25163 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.184+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c72c1a5690 0x55c72c1a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.184+0000 7fdc4bc5d740 4 client.25163 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.184+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/187533232 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c72c100a20 con 0x55c72c1a5690 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.184+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c72c1a5690 0x55c72c1a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.184+0000 7fdc29ffb640 10 client.25163 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.184+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c72c1a5690 0x55c72c1a7b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deecc2e6182fbf49 server_cookie=63b047c605b0bfbc in_seq=0 out_seq=0 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc29ffb640 1 -- 172.21.15.19:0/187533232 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c72c100a20 con 0x55c72c1a5690 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc29ffb640 10 client.25163 handle_command_reply: tid=0 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 client.25163 shutdown 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 msgr2=0x55c72c199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c199ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc340cd8b0 tx=0x7fdc340d0180 comp rx=0 tx=0).stop 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/187533232 shutdown_connections 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c72c1a5690 0x55c72c1a7b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c195370 0x55c72c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc29ffb640 0 client.25163 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/187533232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c199ba0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/187533232 >> 172.21.15.19:0/187533232 conn(0x55c72c18b010 msgr2=0x55c72c18c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/187533232 shutdown_connections 2024-09-19T04:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.185+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/187533232 wait complete. 2024-09-19T04:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.186+0000 7fdc4bc5d740 20 client.25163 trim_cache size 0 max 16384 2024-09-19T04:08:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.214+0000 7fdc4bc5d740 1 Processor -- start 2024-09-19T04:08:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.214+0000 7fdc4bc5d740 1 -- start start 2024-09-19T04:08:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.214+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c18fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.214+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.214+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c18c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.214+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2400fb40 con 0x55c72c195370 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.214+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24011680 con 0x55c72c18f9a0 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.214+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24010210 con 0x55c72c1902e0 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c18c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c18c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48096/0 (socket says 172.21.15.19:48096) 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 -- 172.21.15.19:0/3600095863 learned_addr learned my addr 172.21.15.19:0/3600095863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc4259c640 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c18fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 -- 172.21.15.19:0/3600095863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 msgr2=0x55c72c1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 -- 172.21.15.19:0/3600095863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 msgr2=0x55c72c18fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c18fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 -- 172.21.15.19:0/3600095863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c72c1996e0 con 0x55c72c195370 2024-09-19T04:08:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c18c5d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0104d0 tx=0x7fdc3c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bb4a88554acfa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.215+0000 7fdc2bfff640 1 -- 172.21.15.19:0/3600095863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00eb10 con 0x55c72c195370 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc2bfff640 1 -- 172.21.15.19:0/3600095863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc3c02b030 con 0x55c72c195370 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc2bfff640 1 -- 172.21.15.19:0/3600095863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00e430 con 0x55c72c195370 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/3600095863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 msgr2=0x55c72c18c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c18c5d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0104d0 tx=0x7fdc3c009c90 comp rx=0 tx=0).stop 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/3600095863 shutdown_connections 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c195370 0x55c72c18c5d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c1902e0 0x55c72c1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/3600095863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c18f9a0 0x55c72c18fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/3600095863 >> 172.21.15.19:0/3600095863 conn(0x55c72c18b010 msgr2=0x55c72c18bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/3600095863 shutdown_connections 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.216+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/3600095863 wait complete. 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 1 -- start start 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 1 Processor -- start 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c18f9a0 0x55c72c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c195370 0x55c72c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24010fb0 con 0x55c72c1902e0 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdc240108e0 con 0x55c72c195370 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2400db20 con 0x55c72c18f9a0 2024-09-19T04:08:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4bc5d740 10 client.0 _ll_register_callbacks cb 0x7fdc3409bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc41d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc41d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48104/0 (socket says 172.21.15.19:48104) 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc41d9b640 1 -- 172.21.15.19:0/966965993 learned_addr learned my addr 172.21.15.19:0/966965993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc4259c640 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c18f9a0 0x55c72c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.217+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c195370 0x55c72c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc41d9b640 1 -- 172.21.15.19:0/966965993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c18f9a0 msgr2=0x55c72c19e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c18f9a0 0x55c72c19e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc41d9b640 1 -- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c195370 msgr2=0x55c72c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c195370 0x55c72c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc41d9b640 1 -- 172.21.15.19:0/966965993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c72c1a0020 con 0x55c72c1902e0 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc42d9d640 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c195370 0x55c72c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc41d9b640 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19ea70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009230 tx=0x7fdc3802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b498dbfe6cc86bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc29ffb640 1 -- 172.21.15.19:0/966965993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38035050 con 0x55c72c1902e0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc29ffb640 1 -- 172.21.15.19:0/966965993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fdc380326c0 con 0x55c72c1902e0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/966965993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc3c02b030 con 0x55c72c1902e0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc29ffb640 1 -- 172.21.15.19:0/966965993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc38063050 con 0x55c72c1902e0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc4bc5d740 10 client.15555 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.218+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/966965993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c72c1a1400 con 0x55c72c1902e0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc29ffb640 1 -- 172.21.15.19:0/966965993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc3c02b030 con 0x55c72c1902e0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc29ffb640 1 -- 172.21.15.19:0/966965993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdc380035f0 con 0x55c72c1902e0 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc4bc5d740 10 client.15555 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc4bc5d740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc4bc5d740 20 client.15555 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c72c210ba0 0x55c72c212fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc4bc5d740 4 client.15555 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/966965993 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c72c0fad70 con 0x55c72c210ba0 2024-09-19T04:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc4259c640 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c72c210ba0 0x55c72c212fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc29ffb640 10 client.15555 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.219+0000 7fdc4259c640 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c72c210ba0 0x55c72c212fc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f6704094b02b92 server_cookie=bc4e516a824fd6a1 in_seq=0 out_seq=0 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc29ffb640 1 -- 172.21.15.19:0/966965993 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c72c0fad70 con 0x55c72c210ba0 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc29ffb640 10 client.15555 handle_command_reply: tid=0 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 client.15555 shutdown 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/966965993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 msgr2=0x55c72c19ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19ea70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009230 tx=0x7fdc3802a410 comp rx=0 tx=0).stop 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/966965993 shutdown_connections 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c72c195370 0x55c72c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c72c1902e0 0x55c72c19ea70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55c72c210ba0 0x55c72c212fc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 --2- 172.21.15.19:0/966965993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c72c18f9a0 0x55c72c19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc29ffb640 0 client.15555 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.223+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/966965993 >> 172.21.15.19:0/966965993 conn(0x55c72c18b010 msgr2=0x55c72c1899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/966965993 shutdown_connections 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc4bc5d740 1 -- 172.21.15.19:0/966965993 wait complete. 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc4bc5d740 20 client.15555 trim_cache size 0 max 16384 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc2c0a3410 msgr2=0x7fdc2c0a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc2c0a3410 0x7fdc2c0a58d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006010 tx=0x7fdc38000bc0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc44101b60 msgr2=0x7fdc44075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc44101b60 0x7fdc44075fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400e9c0 tx=0x7fdc3400ee80 comp rx=0 tx=0).stop 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 shutdown_connections 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fdc2c0a3410 0x7fdc2c0a58d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc44103710 0x7fdc44074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc44102d50 0x7fdc440745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 --2- 172.21.15.19:0/317725524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc44101b60 0x7fdc44075fa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 >> 172.21.15.19:0/317725524 conn(0x7fdc440fd2e0 msgr2=0x7fdc44106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 shutdown_connections 2024-09-19T04:08:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:10.224+0000 7fdc48ec3640 1 -- 172.21.15.19:0/317725524 wait complete. 2024-09-19T04:08:10.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:10 smithi111 ceph-mon[35365]: pgmap v397: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T04:08:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:10 smithi019 ceph-mon[30938]: pgmap v397: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T04:08:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:10 smithi196 ceph-mon[35731]: pgmap v397: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T04:08:12.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:12 smithi111 ceph-mon[35365]: pgmap v398: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T04:08:12.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:12 smithi111 ceph-mon[35365]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T04:08:12.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:12 smithi019 ceph-mon[30938]: pgmap v398: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T04:08:12.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:12 smithi019 ceph-mon[30938]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T04:08:12.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:12 smithi196 ceph-mon[35731]: pgmap v398: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T04:08:12.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:12 smithi196 ceph-mon[35731]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T04:08:14.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:14 smithi111 ceph-mon[35365]: pgmap v399: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:08:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:14 smithi019 ceph-mon[30938]: pgmap v399: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:08:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:14 smithi196 ceph-mon[35731]: pgmap v399: 129 pgs: 129 active+clean; 674 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T04:08:15.201 INFO:tasks.workunit.client.0.smithi019.stderr:Updating files: 94% (12224/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T04:08:15.681 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T04:08:15.681 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-19T04:08:15.681 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-19T04:08:15.682 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T04:08:15.683 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-19T04:08:15.683 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T04:08:15.686 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T04:08:15.753 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T04:08:15.755 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T04:08:15.755 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T04:08:16.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:16 smithi111 ceph-mon[35365]: pgmap v400: 129 pgs: 129 active+clean; 675 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T04:08:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:16 smithi019 ceph-mon[30938]: pgmap v400: 129 pgs: 129 active+clean; 675 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T04:08:16.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:16 smithi196 ceph-mon[35731]: pgmap v400: 129 pgs: 129 active+clean; 675 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T04:08:16.462 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T04:08:16.546 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T04:08:16.590 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T04:08:16.591 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T04:08:16.591 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T04:08:16.633 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T04:08:16.636 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T04:08:16.636 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T04:08:16.663 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T04:08:16.663 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T04:08:16.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:08:16.694 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-09-19T04:08:16.695 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-09-19T04:08:16.696 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-09-19T04:08:16.758 INFO:tasks.workunit.client.0.smithi019.stdout:test large (16MB) dio write 2024-09-19T04:08:16.758 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test large (16MB) dio write' 2024-09-19T04:08:16.758 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-09-19T04:08:16.836 INFO:tasks.workunit.client.0.smithi019.stderr:1+0 records in 2024-09-19T04:08:16.837 INFO:tasks.workunit.client.0.smithi019.stderr:1+0 records out 2024-09-19T04:08:16.837 INFO:tasks.workunit.client.0.smithi019.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0766899 s, 219 MB/s 2024-09-19T04:08:16.837 INFO:tasks.workunit.client.0.smithi019.stdout:OK 2024-09-19T04:08:16.838 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo OK 2024-09-19T04:08:16.838 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T04:08:16.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T04:08:16.916 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-09-19T04:08:16.916 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-09-19T04:08:16.950 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:08:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 -- 172.21.15.19:0/1416669130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4102d40 msgr2=0x7f22a4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1416669130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4102d40 0x7f22a4103120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2294009900 tx=0x7f229402a480 comp rx=0 tx=0).stop 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 -- 172.21.15.19:0/1416669130 shutdown_connections 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1416669130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a410c170 0x7f22a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1416669130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4103660 0x7f22a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1416669130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4102d40 0x7f22a4103120 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 -- 172.21.15.19:0/1416669130 >> 172.21.15.19:0/1416669130 conn(0x7f22a40fc7a0 msgr2=0x7f22a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 -- 172.21.15.19:0/1416669130 shutdown_connections 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.508+0000 7f22a9a38640 1 -- 172.21.15.19:0/1416669130 wait complete. 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.509+0000 7f22a9a38640 1 Processor -- start 2024-09-19T04:08:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.509+0000 7f22a9a38640 1 -- start start 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.509+0000 7f22a9a38640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4102d40 0x7f22a4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.509+0000 7f22a9a38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103660 0x7f22a41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.509+0000 7f22a9a38640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a410c170 0x7f22a419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a9a38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4111fb0 con 0x7f22a4103660 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a9a38640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4111e30 con 0x7f22a410c170 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a9a38640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4112130 con 0x7f22a4102d40 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103660 0x7f22a41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103660 0x7f22a41989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49770/0 (socket says 172.21.15.19:49770) 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 -- 172.21.15.19:0/1324910467 learned_addr learned my addr 172.21.15.19:0/1324910467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4102d40 0x7f22a4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a37fe640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a410c170 0x7f22a419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 -- 172.21.15.19:0/1324910467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4102d40 msgr2=0x7f22a4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4102d40 0x7f22a4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 -- 172.21.15.19:0/1324910467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a410c170 msgr2=0x7f22a419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a410c170 0x7f22a419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 -- 172.21.15.19:0/1324910467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a419d440 con 0x7f22a4103660 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a37fe640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a410c170 0x7f22a419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f229a7fc640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103660 0x7f22a41989d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f228c00d740 tx=0x7f228c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a8a36640 1 -- 172.21.15.19:0/1324910467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c017070 con 0x7f22a4103660 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a419d730 con 0x7f22a4103660 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a8a36640 1 -- 172.21.15.19:0/1324910467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228c00de50 con 0x7f22a4103660 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.510+0000 7f22a8a36640 1 -- 172.21.15.19:0/1324910467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c00f620 con 0x7f22a4103660 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.511+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a5010 con 0x7f22a4103660 2024-09-19T04:08:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.511+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a4103120 con 0x7f22a4103660 2024-09-19T04:08:17.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.513+0000 7f22a8a36640 1 -- 172.21.15.19:0/1324910467 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f228c009c30 con 0x7f22a4103660 2024-09-19T04:08:17.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.513+0000 7f22a8a36640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f22900a3240 0x7f22900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:17.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.514+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f22900a3240 0x7f22900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:17.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.514+0000 7f22a8a36640 1 -- 172.21.15.19:0/1324910467 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f228c0cf910 con 0x7f22a4103660 2024-09-19T04:08:17.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.514+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f22900a3240 0x7f22900a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2294009230 tx=0x7f22940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:17.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.518+0000 7f22a8a36640 1 -- 172.21.15.19:0/1324910467 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f228c09c230 con 0x7f22a4103660 2024-09-19T04:08:17.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.730+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22a4107210 con 0x7f22a4103660 2024-09-19T04:08:17.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.730+0000 7f22a8a36640 1 -- 172.21.15.19:0/1324910467 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7f228c09bb30 con 0x7f22a4103660 2024-09-19T04:08:17.733 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:08:17.733 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:08:17.733 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:08:17.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.732+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f22900a3240 msgr2=0x7f22900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:17.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.732+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f22900a3240 0x7f22900a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2294009230 tx=0x7f22940058d0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.732+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103660 msgr2=0x7f22a41989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:17.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.732+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103660 0x7f22a41989d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f228c00d740 tx=0x7f228c00dc00 comp rx=0 tx=0).stop 2024-09-19T04:08:17.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.733+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 shutdown_connections 2024-09-19T04:08:17.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.733+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f22900a3240 0x7f22900a5700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.733+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22a410c170 0x7f22a419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.733+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103660 0x7f22a41989d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.733+0000 7f22a9a38640 1 --2- 172.21.15.19:0/1324910467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4102d40 0x7f22a4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:17.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.733+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 >> 172.21.15.19:0/1324910467 conn(0x7f22a40fc7a0 msgr2=0x7f22a40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:17.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.733+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 shutdown_connections 2024-09-19T04:08:17.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:17.733+0000 7f22a9a38640 1 -- 172.21.15.19:0/1324910467 wait complete. 2024-09-19T04:08:18.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:18 smithi019 ceph-mon[30938]: pgmap v401: 129 pgs: 129 active+clean; 675 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T04:08:18.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:18 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1324910467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:18.404 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T04:08:18.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T04:08:18.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:18 smithi111 ceph-mon[35365]: pgmap v401: 129 pgs: 129 active+clean; 675 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T04:08:18.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:18 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1324910467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:18 smithi196 ceph-mon[35731]: pgmap v401: 129 pgs: 129 active+clean; 675 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T04:08:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:18 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1324910467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 -- 172.21.15.19:0/3378798468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330102570 msgr2=0x7f03301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 --2- 172.21.15.19:0/3378798468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330102570 0x7f03301029b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0318009900 tx=0x7f031802a480 comp rx=0 tx=0).stop 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 -- 172.21.15.19:0/3378798468 shutdown_connections 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 --2- 172.21.15.19:0/3378798468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330102ef0 0x7f033010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 --2- 172.21.15.19:0/3378798468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330102570 0x7f03301029b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 --2- 172.21.15.19:0/3378798468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0330100e20 0x7f0330101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 -- 172.21.15.19:0/3378798468 >> 172.21.15.19:0/3378798468 conn(0x7f03300fc7d0 msgr2=0x7f03300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 -- 172.21.15.19:0/3378798468 shutdown_connections 2024-09-19T04:08:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.142+0000 7f03364a5640 1 -- 172.21.15.19:0/3378798468 wait complete. 2024-09-19T04:08:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f03364a5640 1 Processor -- start 2024-09-19T04:08:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f03364a5640 1 -- start start 2024-09-19T04:08:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f03364a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330100e20 0x7f033019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f03364a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0330102570 0x7f033019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f03364a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330102ef0 0x7f033019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f03364a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0330110fa0 con 0x7f0330100e20 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f03364a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0330110e20 con 0x7f0330102ef0 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f03364a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0330111120 con 0x7f0330102570 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f032ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330100e20 0x7f033019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f032ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330100e20 0x7f033019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49786/0 (socket says 172.21.15.19:49786) 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f032ffff640 1 -- 172.21.15.19:0/3722804903 learned_addr learned my addr 172.21.15.19:0/3722804903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.143+0000 7f0334a42640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330102ef0 0x7f033019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f0327fff640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0330102570 0x7f033019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032ffff640 1 -- 172.21.15.19:0/3722804903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0330102570 msgr2=0x7f033019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032ffff640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0330102570 0x7f033019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032ffff640 1 -- 172.21.15.19:0/3722804903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330102ef0 msgr2=0x7f033019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032ffff640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330102ef0 0x7f033019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032ffff640 1 -- 172.21.15.19:0/3722804903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033019f6f0 con 0x7f0330100e20 2024-09-19T04:08:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032ffff640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330100e20 0x7f033019a740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f032000d950 tx=0x7f032000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032dffb640 1 -- 172.21.15.19:0/3722804903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320017070 con 0x7f0330100e20 2024-09-19T04:08:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032dffb640 1 -- 172.21.15.19:0/3722804903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0320009c30 con 0x7f0330100e20 2024-09-19T04:08:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f03364a5640 1 -- 172.21.15.19:0/3722804903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033019f9e0 con 0x7f0330100e20 2024-09-19T04:08:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f032dffb640 1 -- 172.21.15.19:0/3722804903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032000f6b0 con 0x7f0330100e20 2024-09-19T04:08:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.144+0000 7f03364a5640 1 -- 172.21.15.19:0/3722804903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301a72c0 con 0x7f0330100e20 2024-09-19T04:08:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.146+0000 7f032dffb640 1 -- 172.21.15.19:0/3722804903 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f03200109d0 con 0x7f0330100e20 2024-09-19T04:08:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f033887e740 1 Processor -- start 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f033887e740 1 -- start start 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f033887e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f5fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f033887e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f033887e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5f6ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f033887e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ee5ec4160 con 0x564ee5f60170 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f033887e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564ee5e6b720 con 0x564ee5f5f850 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f033887e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ee5ec0120 con 0x564ee5f6aa30 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f032ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f032ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49792/0 (socket says 172.21.15.19:49792) 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f032ffff640 1 -- 172.21.15.19:0/226294286 learned_addr learned my addr 172.21.15.19:0/226294286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f0334a42640 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f5fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f0327fff640 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5f6ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f032ffff640 1 -- 172.21.15.19:0/226294286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 msgr2=0x564ee5f6ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f032ffff640 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5f6ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f032ffff640 1 -- 172.21.15.19:0/226294286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 msgr2=0x564ee5f5fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f032ffff640 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f5fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f032ffff640 1 -- 172.21.15.19:0/226294286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee5c35ba0 con 0x564ee5f60170 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.147+0000 7f0334a42640 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f5fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.148+0000 7f032dffb640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f031c0a32c0 0x7f031c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.148+0000 7f032ffff640 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6a4f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f03200d5c10 tx=0x7f03200d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0c91d81444b401 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.148+0000 7f0334a42640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f031c0a32c0 0x7f031c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.148+0000 7f03267fc640 1 -- 172.21.15.19:0/226294286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320097dc0 con 0x564ee5f60170 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.148+0000 7f032dffb640 1 -- 172.21.15.19:0/3722804903 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f03200cf8a0 con 0x7f0330100e20 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.148+0000 7f03267fc640 1 -- 172.21.15.19:0/226294286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032009ce50 con 0x564ee5f60170 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.148+0000 7f03267fc640 1 -- 172.21.15.19:0/226294286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03200b83c0 con 0x564ee5f60170 2024-09-19T04:08:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 -- 172.21.15.19:0/226294286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 msgr2=0x564ee5f6a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f0334a42640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f031c0a32c0 0x7f031c0a5780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0328009fd0 tx=0x7f0328009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6a4f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f03200d5c10 tx=0x7f03200d7690 comp rx=0 tx=0).stop 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 -- 172.21.15.19:0/226294286 shutdown_connections 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5f6ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6a4f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 --2- 172.21.15.19:0/226294286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f5fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 -- 172.21.15.19:0/226294286 >> 172.21.15.19:0/226294286 conn(0x564ee5f5bde0 msgr2=0x564ee5f5c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 -- 172.21.15.19:0/226294286 shutdown_connections 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 -- 172.21.15.19:0/226294286 wait complete. 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 -- start start 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.149+0000 7f033887e740 1 Processor -- start 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f033887e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f5f850 0x564ee5f678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f033887e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f60170 0x564ee5f67e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f033887e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f6aa30 0x564ee5f68360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f033887e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ee5f6d600 con 0x564ee5f6aa30 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f033887e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564ee5e4c900 con 0x564ee5f60170 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f033887e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564ee5f6d780 con 0x564ee5f5f850 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f033887e740 10 client.0 _ll_register_callbacks cb 0x564ee5e4a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f6aa30 0x564ee5f68360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f6aa30 0x564ee5f68360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49800/0 (socket says 172.21.15.19:49800) 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 -- 172.21.15.19:0/2423048057 learned_addr learned my addr 172.21.15.19:0/2423048057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f032ffff640 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f5f850 0x564ee5f678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0327fff640 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f60170 0x564ee5f67e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 -- 172.21.15.19:0/2423048057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f5f850 msgr2=0x564ee5f678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f5f850 0x564ee5f678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 -- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f60170 msgr2=0x564ee5f67e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f60170 0x564ee5f67e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 -- 172.21.15.19:0/2423048057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee5e8cc50 con 0x564ee5f6aa30 2024-09-19T04:08:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0327fff640 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f60170 0x564ee5f67e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.150+0000 7f0334a42640 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f6aa30 0x564ee5f68360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0328011d80 tx=0x7f0328004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d344a2df31329aa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f030f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f030f7fe640 1 -- 172.21.15.19:0/2423048057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032800fe60 con 0x564ee5f6aa30 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 1 -- 172.21.15.19:0/2423048057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f032009ce50 con 0x564ee5f6aa30 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f030f7fe640 1 -- 172.21.15.19:0/2423048057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f032801f9b0 con 0x564ee5f6aa30 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f030f7fe640 1 -- 172.21.15.19:0/2423048057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0328031050 con 0x564ee5f6aa30 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 10 client.15573 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 1 -- 172.21.15.19:0/2423048057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ee5f792e0 con 0x564ee5f6aa30 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f030f7fe640 1 -- 172.21.15.19:0/2423048057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032009ce50 con 0x564ee5f6aa30 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f030f7fe640 1 -- 172.21.15.19:0/2423048057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0328026020 con 0x564ee5f6aa30 2024-09-19T04:08:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 10 client.15573 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 20 client.15573 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x564ee5f7a7a0 0x564ee5f7cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 4 client.15573 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.151+0000 7f033887e740 1 -- 172.21.15.19:0/2423048057 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ee5ed1890 con 0x564ee5f7a7a0 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.152+0000 7f032ffff640 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x564ee5f7a7a0 0x564ee5f7cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.152+0000 7f032ffff640 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x564ee5f7a7a0 0x564ee5f7cbb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=687c2dc03f502ceb server_cookie=17925ae634d293f6 in_seq=0 out_seq=0 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.152+0000 7f030f7fe640 10 client.15573 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f030f7fe640 1 -- 172.21.15.19:0/2423048057 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ee5ed1890 con 0x564ee5f7a7a0 2024-09-19T04:08:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f030f7fe640 10 client.15573 handle_command_reply: tid=0 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 client.15573 shutdown 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 -- 172.21.15.19:0/2423048057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f6aa30 msgr2=0x564ee5f68360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f6aa30 0x564ee5f68360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0328011d80 tx=0x7f0328004ab0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 -- 172.21.15.19:0/2423048057 shutdown_connections 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f6aa30 0x564ee5f68360 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f60170 0x564ee5f67e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x564ee5f7a7a0 0x564ee5f7cbb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 --2- 172.21.15.19:0/2423048057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f5f850 0x564ee5f678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f030f7fe640 0 client.15573 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 -- 172.21.15.19:0/2423048057 >> 172.21.15.19:0/2423048057 conn(0x564ee5f5bde0 msgr2=0x564ee5f61b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 -- 172.21.15.19:0/2423048057 shutdown_connections 2024-09-19T04:08:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 1 -- 172.21.15.19:0/2423048057 wait complete. 2024-09-19T04:08:19.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.153+0000 7f033887e740 20 client.15573 trim_cache size 0 max 16384 2024-09-19T04:08:19.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.170+0000 7f033887e740 1 Processor -- start 2024-09-19T04:08:19.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.170+0000 7f033887e740 1 -- start start 2024-09-19T04:08:19.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f033887e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f033887e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f82480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f033887e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5f829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f033887e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03301b0f60 con 0x564ee5f60170 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f033887e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03301b2aa0 con 0x564ee5f5f850 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f033887e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03301b1630 con 0x564ee5f6aa30 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f82480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f82480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49810/0 (socket says 172.21.15.19:49810) 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 -- 172.21.15.19:0/1104444370 learned_addr learned my addr 172.21.15.19:0/1104444370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0334a42640 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5f829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f032ffff640 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 -- 172.21.15.19:0/1104444370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 msgr2=0x564ee5f829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5f829c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 -- 172.21.15.19:0/1104444370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 msgr2=0x564ee5f67fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:19.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f67fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 -- 172.21.15.19:0/1104444370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee5f5bc10 con 0x564ee5f60170 2024-09-19T04:08:19.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0327fff640 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f82480 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0318009900 tx=0x7f03180042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237ac51494571d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.171+0000 7f0325ffb640 1 -- 172.21.15.19:0/1104444370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318035070 con 0x564ee5f60170 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f0325ffb640 1 -- 172.21.15.19:0/1104444370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0318005520 con 0x564ee5f60170 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f0325ffb640 1 -- 172.21.15.19:0/1104444370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318003850 con 0x564ee5f60170 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 -- 172.21.15.19:0/1104444370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 msgr2=0x564ee5f82480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f82480 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0318009900 tx=0x7f03180042d0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 -- 172.21.15.19:0/1104444370 shutdown_connections 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5f829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f82480 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 --2- 172.21.15.19:0/1104444370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f67fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 -- 172.21.15.19:0/1104444370 >> 172.21.15.19:0/1104444370 conn(0x564ee5f5bde0 msgr2=0x564ee5f678a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 -- 172.21.15.19:0/1104444370 shutdown_connections 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 -- 172.21.15.19:0/1104444370 wait complete. 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.172+0000 7f033887e740 1 -- start start 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f033887e740 1 Processor -- start 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f033887e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f6f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f033887e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f033887e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5fdd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f033887e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03301b23d0 con 0x564ee5f60170 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f033887e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03301b1d00 con 0x564ee5f5f850 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f033887e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03301aef40 con 0x564ee5f6aa30 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f033887e740 10 client.0 _ll_register_callbacks cb 0x7f0328009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f0327fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f0327fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49816/0 (socket says 172.21.15.19:49816) 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f0327fff640 1 -- 172.21.15.19:0/991849647 learned_addr learned my addr 172.21.15.19:0/991849647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f032ffff640 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f6f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.173+0000 7f0334a42640 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5fdd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f0327fff640 1 -- 172.21.15.19:0/991849647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 msgr2=0x564ee5fdd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f0327fff640 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5fdd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f0327fff640 1 -- 172.21.15.19:0/991849647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 msgr2=0x564ee5f6f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f0327fff640 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f6f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f0327fff640 1 -- 172.21.15.19:0/991849647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee5f573f0 con 0x564ee5f60170 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f0334a42640 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5fdd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f0327fff640 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6f880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f031802aa70 tx=0x7f0318005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404094e39ede051 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f030f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f030f7fe640 1 -- 172.21.15.19:0/991849647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318035050 con 0x564ee5f60170 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f030f7fe640 1 -- 172.21.15.19:0/991849647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f0318005230 con 0x564ee5f60170 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f033887e740 1 -- 172.21.15.19:0/991849647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0318005520 con 0x564ee5f60170 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f030f7fe640 1 -- 172.21.15.19:0/991849647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0318056050 con 0x564ee5f60170 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f033887e740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f033887e740 1 -- 172.21.15.19:0/991849647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ee5fe5d90 con 0x564ee5f60170 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.174+0000 7f030f7fe640 1 -- 172.21.15.19:0/991849647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0318005520 con 0x564ee5f60170 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f030f7fe640 1 -- 172.21.15.19:0/991849647 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0318042020 con 0x564ee5f60170 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f033887e740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f033887e740 10 client.15579 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f033887e740 20 client.15579 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f033887e740 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x564ee5fe64a0 0x564ee5fe88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f033887e740 4 client.15579 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T04:08:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f033887e740 1 -- 172.21.15.19:0/991849647 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x564ee5ecbbe0 con 0x564ee5fe64a0 2024-09-19T04:08:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f032ffff640 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x564ee5fe64a0 0x564ee5fe88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f032ffff640 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x564ee5fe64a0 0x564ee5fe88c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15bc228b8e0fb6c5 server_cookie=3d12d90d4c856859 in_seq=0 out_seq=0 2024-09-19T04:08:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.175+0000 7f030f7fe640 10 client.15579 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.770+0000 7f030f7fe640 1 -- 172.21.15.19:0/991849647 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564ee5ecbbe0 con 0x564ee5fe64a0 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.770+0000 7f030f7fe640 10 client.15579 handle_command_reply: tid=0 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.770+0000 7f033887e740 1 client.15579 shutdown 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 -- 172.21.15.19:0/991849647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 msgr2=0x564ee5f6f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6f880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f031802aa70 tx=0x7f0318005bc0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 -- 172.21.15.19:0/991849647 shutdown_connections 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x564ee5fe64a0 0x564ee5fe88c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564ee5f6aa30 0x564ee5fdd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee5f60170 0x564ee5f6f880 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 --2- 172.21.15.19:0/991849647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564ee5f5f850 0x564ee5f6f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f030f7fe640 0 client.15579 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 -- 172.21.15.19:0/991849647 >> 172.21.15.19:0/991849647 conn(0x564ee5f5bde0 msgr2=0x564ee5f5aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 -- 172.21.15.19:0/991849647 shutdown_connections 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 1 -- 172.21.15.19:0/991849647 wait complete. 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.771+0000 7f033887e740 20 client.15579 trim_cache size 0 max 16384 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 29, 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24476, 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 6, 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "75de194c": { 2024-09-19T04:08:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 6, 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.3, 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:08:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-19T04:08:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:08:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:19.777 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 -- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f031c0a32c0 msgr2=0x7f031c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f031c0a32c0 0x7f031c0a5780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0328009fd0 tx=0x7f0328009500 comp rx=0 tx=0).stop 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 -- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330100e20 msgr2=0x7f033019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330100e20 0x7f033019a740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f032000d950 tx=0x7f032000de10 comp rx=0 tx=0).stop 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 -- 172.21.15.19:0/3722804903 shutdown_connections 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f031c0a32c0 0x7f031c0a5780 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330102ef0 0x7f033019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0330102570 0x7f033019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 --2- 172.21.15.19:0/3722804903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0330100e20 0x7f033019a740 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 -- 172.21.15.19:0/3722804903 >> 172.21.15.19:0/3722804903 conn(0x7f03300fc7d0 msgr2=0x7f033006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 -- 172.21.15.19:0/3722804903 shutdown_connections 2024-09-19T04:08:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:19.772+0000 7f03364a5640 1 -- 172.21.15.19:0/3722804903 wait complete. 2024-09-19T04:08:20.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:20 smithi111 ceph-mon[35365]: pgmap v402: 129 pgs: 129 active+clean; 13 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-09-19T04:08:20.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:20 smithi019 ceph-mon[30938]: pgmap v402: 129 pgs: 129 active+clean; 13 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-09-19T04:08:20.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:20 smithi196 ceph-mon[35731]: pgmap v402: 129 pgs: 129 active+clean; 13 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-09-19T04:08:20.489 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '75de194c', quiesce took 3.8 seconds. Will release after: 16.55 2024-09-19T04:08:22.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:22 smithi111 ceph-mon[35365]: pgmap v403: 129 pgs: 129 active+clean; 17 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-09-19T04:08:22.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:22 smithi019 ceph-mon[30938]: pgmap v403: 129 pgs: 129 active+clean; 17 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-09-19T04:08:22.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:22 smithi196 ceph-mon[35731]: pgmap v403: 129 pgs: 129 active+clean; 17 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-09-19T04:08:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 -- 172.21.15.19:0/692937260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 msgr2=0x7fe618100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 --2- 172.21.15.19:0/692937260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 0x7fe618100e40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe608009900 tx=0x7fe60802a410 comp rx=0 tx=0).stop 2024-09-19T04:08:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 -- 172.21.15.19:0/692937260 shutdown_connections 2024-09-19T04:08:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 --2- 172.21.15.19:0/692937260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6181025f0 0x7fe618108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 --2- 172.21.15.19:0/692937260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe618101c30 0x7fe6181020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 --2- 172.21.15.19:0/692937260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 0x7fe618100e40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 -- 172.21.15.19:0/692937260 >> 172.21.15.19:0/692937260 conn(0x7fe6180fc1e0 msgr2=0x7fe6180fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 -- 172.21.15.19:0/692937260 shutdown_connections 2024-09-19T04:08:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.329+0000 7fe61dea5640 1 -- 172.21.15.19:0/692937260 wait complete. 2024-09-19T04:08:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.330+0000 7fe61dea5640 1 Processor -- start 2024-09-19T04:08:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.330+0000 7fe61dea5640 1 -- start start 2024-09-19T04:08:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.330+0000 7fe61dea5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 0x7fe618197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.330+0000 7fe61dea5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe618101c30 0x7fe618198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.330+0000 7fe61dea5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6181025f0 0x7fe61819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.330+0000 7fe61dea5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe61810c6c0 con 0x7fe618100a40 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe61dea5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe61810c540 con 0x7fe6181025f0 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe61dea5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe61810c840 con 0x7fe618101c30 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 0x7fe618197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 0x7fe618197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41834/0 (socket says 172.21.15.19:41834) 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 -- 172.21.15.19:0/2248824029 learned_addr learned my addr 172.21.15.19:0/2248824029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe617fff640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6181025f0 0x7fe61819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe616ffd640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe618101c30 0x7fe618198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 -- 172.21.15.19:0/2248824029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe618101c30 msgr2=0x7fe618198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe618101c30 0x7fe618198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 -- 172.21.15.19:0/2248824029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6181025f0 msgr2=0x7fe61819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6181025f0 0x7fe61819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 -- 172.21.15.19:0/2248824029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61819fa30 con 0x7fe618100a40 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.331+0000 7fe6177fe640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 0x7fe618197e10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe608031610 tx=0x7fe608004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.332+0000 7fe614ff9640 1 -- 172.21.15.19:0/2248824029 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60803f030 con 0x7fe618100a40 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.332+0000 7fe61dea5640 1 -- 172.21.15.19:0/2248824029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61819fcc0 con 0x7fe618100a40 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.332+0000 7fe614ff9640 1 -- 172.21.15.19:0/2248824029 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe608003ad0 con 0x7fe618100a40 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.332+0000 7fe614ff9640 1 -- 172.21.15.19:0/2248824029 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608035070 con 0x7fe618100a40 2024-09-19T04:08:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.332+0000 7fe61dea5640 1 -- 172.21.15.19:0/2248824029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6181a01b0 con 0x7fe618100a40 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.334+0000 7fe614ff9640 1 -- 172.21.15.19:0/2248824029 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe608042030 con 0x7fe618100a40 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.334+0000 7fe620c43740 1 Processor -- start 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe620c43740 1 -- start start 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe620c43740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b878da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe620c43740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8792e0 0x56518b87dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe620c43740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b880830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe620c43740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56518b7dc2f0 con 0x56518b8792e0 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe620c43740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56518b784270 con 0x56518b87e370 2024-09-19T04:08:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe620c43740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56518b7d82b0 con 0x56518b8789a0 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8792e0 0x56518b87dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe617fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b878da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe616ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b880830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe614ff9640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe6000a32f0 0x7fe6000a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8792e0 0x56518b87dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41848/0 (socket says 172.21.15.19:41848) 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe617fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b878da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38378/0 (socket says 172.21.15.19:38378) 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe6177fe640 1 -- 172.21.15.19:0/634337471 learned_addr learned my addr 172.21.15.19:0/634337471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe617fff640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe6000a32f0 0x7fe6000a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe614ff9640 1 -- 172.21.15.19:0/2248824029 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fe6080f02a0 con 0x7fe618100a40 2024-09-19T04:08:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe6177fe640 1 -- 172.21.15.19:0/634337471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 msgr2=0x56518b878da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe6177fe640 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b878da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe6177fe640 1 -- 172.21.15.19:0/634337471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 msgr2=0x56518b880830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe6177fe640 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b880830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe6177fe640 1 -- 172.21.15.19:0/634337471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56518b783340 con 0x56518b8792e0 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe616ffd640 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b880830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.335+0000 7fe617fff640 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b878da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.336+0000 7fe6177fe640 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8792e0 0x56518b87dda0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe6080b8f40 tx=0x7fe60802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e2df01adb5f5cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.336+0000 7fe5fdffb640 1 -- 172.21.15.19:0/634337471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080ede60 con 0x56518b8792e0 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.336+0000 7fe5fdffb640 1 -- 172.21.15.19:0/634337471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60803be40 con 0x56518b8792e0 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.336+0000 7fe617fff640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe6000a32f0 0x7fe6000a57b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe610005c40 tx=0x7fe610004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.336+0000 7fe5fdffb640 1 -- 172.21.15.19:0/634337471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608049070 con 0x56518b8792e0 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 -- 172.21.15.19:0/634337471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8792e0 msgr2=0x56518b87dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8792e0 0x56518b87dda0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe6080b8f40 tx=0x7fe60802a410 comp rx=0 tx=0).stop 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 -- 172.21.15.19:0/634337471 shutdown_connections 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b880830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8792e0 0x56518b87dda0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 --2- 172.21.15.19:0/634337471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b878da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 -- 172.21.15.19:0/634337471 >> 172.21.15.19:0/634337471 conn(0x56518b874010 msgr2=0x56518b874410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 -- 172.21.15.19:0/634337471 shutdown_connections 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 -- 172.21.15.19:0/634337471 wait complete. 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 -- start start 2024-09-19T04:08:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.337+0000 7fe620c43740 1 Processor -- start 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe620c43740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe620c43740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8792e0 0x56518b882e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe620c43740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b889ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe620c43740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56518b881100 con 0x56518b8789a0 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe620c43740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56518b765290 con 0x56518b87e370 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe620c43740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56518b881280 con 0x56518b8792e0 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe620c43740 10 client.0 _ll_register_callbacks cb 0x56518b796140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41850/0 (socket says 172.21.15.19:41850) 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 -- 172.21.15.19:0/1238803936 learned_addr learned my addr 172.21.15.19:0/1238803936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe617fff640 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b889ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe616ffd640 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8792e0 0x56518b882e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 -- 172.21.15.19:0/1238803936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8792e0 msgr2=0x56518b882e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8792e0 0x56518b882e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 -- 172.21.15.19:0/1238803936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 msgr2=0x56518b889ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b889ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 -- 172.21.15.19:0/1238803936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56518b4ef210 con 0x56518b8789a0 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe6177fe640 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882940 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe6080c4c00 tx=0x7fe60803b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c39f162af4682c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe5f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1238803936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608049040 con 0x56518b8789a0 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1238803936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fe6080b8be0 con 0x56518b8789a0 2024-09-19T04:08:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.338+0000 7fe620c43740 1 -- 172.21.15.19:0/1238803936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56518b7e9a20 con 0x56518b8789a0 2024-09-19T04:08:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1238803936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6080ede60 con 0x56518b8789a0 2024-09-19T04:08:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe620c43740 10 client.15591 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe620c43740 1 -- 172.21.15.19:0/1238803936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56518b5bb620 con 0x56518b8789a0 2024-09-19T04:08:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1238803936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56518b7e9a20 con 0x56518b8789a0 2024-09-19T04:08:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1238803936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe6080c4ca0 con 0x56518b8789a0 2024-09-19T04:08:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe620c43740 10 client.15591 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe620c43740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:08:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe620c43740 20 client.15591 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe620c43740 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56518b88e3d0 0x56518b8907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe620c43740 4 client.15591 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:08:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe620c43740 1 -- 172.21.15.19:0/1238803936 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56518b7e3d70 con 0x56518b88e3d0 2024-09-19T04:08:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.339+0000 7fe616ffd640 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56518b88e3d0 0x56518b8907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.340+0000 7fe616ffd640 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56518b88e3d0 0x56518b8907c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c761b226a46565 server_cookie=b653017b2dc36869 in_seq=0 out_seq=0 2024-09-19T04:08:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.340+0000 7fe5f6ffd640 10 client.15591 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.340+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1238803936 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56518b7e3d70 con 0x56518b88e3d0 2024-09-19T04:08:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.340+0000 7fe5f6ffd640 10 client.15591 handle_command_reply: tid=0 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 client.15591 shutdown 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 -- 172.21.15.19:0/1238803936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 msgr2=0x56518b882940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882940 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe6080c4c00 tx=0x7fe60803b690 comp rx=0 tx=0).stop 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 -- 172.21.15.19:0/1238803936 shutdown_connections 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b87e370 0x56518b889ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56518b88e3d0 0x56518b8907c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8792e0 0x56518b882e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 --2- 172.21.15.19:0/1238803936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882940 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe5f6ffd640 0 client.15591 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 -- 172.21.15.19:0/1238803936 >> 172.21.15.19:0/1238803936 conn(0x56518b874010 msgr2=0x56518b875820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 -- 172.21.15.19:0/1238803936 shutdown_connections 2024-09-19T04:08:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 1 -- 172.21.15.19:0/1238803936 wait complete. 2024-09-19T04:08:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.341+0000 7fe620c43740 20 client.15591 trim_cache size 0 max 16384 2024-09-19T04:08:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe620c43740 1 Processor -- start 2024-09-19T04:08:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe620c43740 1 -- start start 2024-09-19T04:08:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe620c43740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe620c43740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 0x56518b8739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe620c43740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b87e370 0x56518b889de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe620c43740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181aa720 con 0x56518b8789a0 2024-09-19T04:08:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe620c43740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181ac260 con 0x56518b8792e0 2024-09-19T04:08:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe620c43740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181aadf0 con 0x56518b87e370 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41860/0 (socket says 172.21.15.19:41860) 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe6177fe640 1 -- 172.21.15.19:0/311974509 learned_addr learned my addr 172.21.15.19:0/311974509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe616ffd640 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 0x56518b8739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe617fff640 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b87e370 0x56518b889de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe6177fe640 1 -- 172.21.15.19:0/311974509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b87e370 msgr2=0x56518b889de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe6177fe640 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b87e370 0x56518b889de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe6177fe640 1 -- 172.21.15.19:0/311974509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 msgr2=0x56518b8739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe6177fe640 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 0x56518b8739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.356+0000 7fe6177fe640 1 -- 172.21.15.19:0/311974509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56518b890b30 con 0x56518b8789a0 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe6177fe640 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882f10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe6080bc500 tx=0x7fe60803b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109fde039cd24a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe5fd7fa640 1 -- 172.21.15.19:0/311974509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608049070 con 0x56518b8789a0 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe5fd7fa640 1 -- 172.21.15.19:0/311974509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6080bc570 con 0x56518b8789a0 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe5fd7fa640 1 -- 172.21.15.19:0/311974509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080b8d60 con 0x56518b8789a0 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 -- 172.21.15.19:0/311974509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 msgr2=0x56518b882f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882f10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe6080bc500 tx=0x7fe60803b700 comp rx=0 tx=0).stop 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 -- 172.21.15.19:0/311974509 shutdown_connections 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b87e370 0x56518b889de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 0x56518b8739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 --2- 172.21.15.19:0/311974509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b8789a0 0x56518b882f10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 -- 172.21.15.19:0/311974509 >> 172.21.15.19:0/311974509 conn(0x56518b874010 msgr2=0x56518b874ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 -- 172.21.15.19:0/311974509 shutdown_connections 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.357+0000 7fe620c43740 1 -- 172.21.15.19:0/311974509 wait complete. 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 1 -- start start 2024-09-19T04:08:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 1 Processor -- start 2024-09-19T04:08:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b887090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 0x56518b8875d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b87e370 0x56518b888590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181abb90 con 0x56518b87e370 2024-09-19T04:08:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181ab4c0 con 0x56518b8792e0 2024-09-19T04:08:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181a8700 con 0x56518b8789a0 2024-09-19T04:08:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe620c43740 10 client.0 _ll_register_callbacks cb 0x7fe60c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe617fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b87e370 0x56518b888590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe617fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b87e370 0x56518b888590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41868/0 (socket says 172.21.15.19:41868) 2024-09-19T04:08:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe617fff640 1 -- 172.21.15.19:0/1265156479 learned_addr learned my addr 172.21.15.19:0/1265156479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe616ffd640 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 0x56518b8875d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.358+0000 7fe6177fe640 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b887090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe617fff640 1 -- 172.21.15.19:0/1265156479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 msgr2=0x56518b887090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe617fff640 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b887090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe617fff640 1 -- 172.21.15.19:0/1265156479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 msgr2=0x56518b8875d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe617fff640 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 0x56518b8875d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe617fff640 1 -- 172.21.15.19:0/1265156479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56518b888ba0 con 0x56518b87e370 2024-09-19T04:08:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe617fff640 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b87e370 0x56518b888590 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe6100039f0 tx=0x7fe610004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509b2ae833d07857 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe5f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1265156479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610004460 con 0x56518b87e370 2024-09-19T04:08:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1265156479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fe610004600 con 0x56518b87e370 2024-09-19T04:08:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe620c43740 1 -- 172.21.15.19:0/1265156479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6080bc570 con 0x56518b87e370 2024-09-19T04:08:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1265156479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe610034050 con 0x56518b87e370 2024-09-19T04:08:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe620c43740 10 client.15597 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe620c43740 1 -- 172.21.15.19:0/1265156479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56518b889f80 con 0x56518b87e370 2024-09-19T04:08:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.359+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1265156479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6080bc570 con 0x56518b87e370 2024-09-19T04:08:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1265156479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe61001f040 con 0x56518b87e370 2024-09-19T04:08:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe620c43740 10 client.15597 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe620c43740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:08:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe620c43740 20 client.15597 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe620c43740 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56518b8f96b0 0x56518b8fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe620c43740 4 client.15597 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:08:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe620c43740 1 -- 172.21.15.19:0/1265156479 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56518b760900 con 0x56518b8f96b0 2024-09-19T04:08:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe6177fe640 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56518b8f96b0 0x56518b8fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe5f6ffd640 10 client.15597 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.360+0000 7fe6177fe640 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56518b8f96b0 0x56518b8fbad0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a49946d403b59f5 server_cookie=3d46511331d12c4a in_seq=0 out_seq=0 2024-09-19T04:08:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe5f6ffd640 1 -- 172.21.15.19:0/1265156479 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56518b760900 con 0x56518b8f96b0 2024-09-19T04:08:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe5f6ffd640 10 client.15597 handle_command_reply: tid=0 2024-09-19T04:08:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 client.15597 shutdown 2024-09-19T04:08:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 -- 172.21.15.19:0/1265156479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b87e370 msgr2=0x56518b888590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b87e370 0x56518b888590 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe6100039f0 tx=0x7fe610004a40 comp rx=0 tx=0).stop 2024-09-19T04:08:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 -- 172.21.15.19:0/1265156479 shutdown_connections 2024-09-19T04:08:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56518b87e370 0x56518b888590 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56518b8792e0 0x56518b8875d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56518b8f96b0 0x56518b8fbad0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 --2- 172.21.15.19:0/1265156479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56518b8789a0 0x56518b887090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 -- 172.21.15.19:0/1265156479 >> 172.21.15.19:0/1265156479 conn(0x56518b874010 msgr2=0x56518b8756e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe5f6ffd640 0 client.15597 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 -- 172.21.15.19:0/1265156479 shutdown_connections 2024-09-19T04:08:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 1 -- 172.21.15.19:0/1265156479 wait complete. 2024-09-19T04:08:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.372+0000 7fe620c43740 20 client.15597 trim_cache size 0 max 16384 2024-09-19T04:08:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:08:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:08:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:08:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:08:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 -- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe6000a32f0 msgr2=0x7fe6000a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe6000a32f0 0x7fe6000a57b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe610005c40 tx=0x7fe610004ab0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 -- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 msgr2=0x7fe618197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 0x7fe618197e10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe608031610 tx=0x7fe608004270 comp rx=0 tx=0).stop 2024-09-19T04:08:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 -- 172.21.15.19:0/2248824029 shutdown_connections 2024-09-19T04:08:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe6000a32f0 0x7fe6000a57b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6181025f0 0x7fe61819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe618101c30 0x7fe618198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 --2- 172.21.15.19:0/2248824029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe618100a40 0x7fe618197e10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 -- 172.21.15.19:0/2248824029 >> 172.21.15.19:0/2248824029 conn(0x7fe6180fc1e0 msgr2=0x7fe6180fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 -- 172.21.15.19:0/2248824029 shutdown_connections 2024-09-19T04:08:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:23.373+0000 7fe61dea5640 1 -- 172.21.15.19:0/2248824029 wait complete. 2024-09-19T04:08:24.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:24 smithi111 ceph-mon[35365]: pgmap v404: 129 pgs: 129 active+clean; 17 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-09-19T04:08:24.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:24 smithi019 ceph-mon[30938]: pgmap v404: 129 pgs: 129 active+clean; 17 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-09-19T04:08:24.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:24 smithi196 ceph-mon[35731]: pgmap v404: 129 pgs: 129 active+clean; 17 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-09-19T04:08:26.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:26 smithi111 ceph-mon[35365]: pgmap v405: 129 pgs: 129 active+clean; 699 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-09-19T04:08:26.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:26 smithi019 ceph-mon[30938]: pgmap v405: 129 pgs: 129 active+clean; 699 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-09-19T04:08:26.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:26 smithi196 ceph-mon[35731]: pgmap v405: 129 pgs: 129 active+clean; 699 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-09-19T04:08:28.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:28 smithi111 ceph-mon[35365]: pgmap v406: 129 pgs: 129 active+clean; 699 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-19T04:08:28.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:28 smithi019 ceph-mon[30938]: pgmap v406: 129 pgs: 129 active+clean; 699 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-19T04:08:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:28 smithi196 ceph-mon[35731]: pgmap v406: 129 pgs: 129 active+clean; 699 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-19T04:08:30.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:30 smithi111 ceph-mon[35365]: pgmap v407: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-09-19T04:08:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:30 smithi019 ceph-mon[30938]: pgmap v407: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-09-19T04:08:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:30 smithi196 ceph-mon[35731]: pgmap v407: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-09-19T04:08:32.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:32 smithi111 ceph-mon[35365]: pgmap v408: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 344 KiB/s wr, 13 op/s 2024-09-19T04:08:32.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:32 smithi019 ceph-mon[30938]: pgmap v408: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 344 KiB/s wr, 13 op/s 2024-09-19T04:08:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:32 smithi196 ceph-mon[35731]: pgmap v408: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 344 KiB/s wr, 13 op/s 2024-09-19T04:08:34.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:34 smithi111 ceph-mon[35365]: pgmap v409: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-09-19T04:08:34.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:34 smithi019 ceph-mon[30938]: pgmap v409: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-09-19T04:08:34.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:34 smithi196 ceph-mon[35731]: pgmap v409: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-09-19T04:08:36.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:36 smithi111 ceph-mon[35365]: pgmap v410: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T04:08:36.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:36 smithi019 ceph-mon[30938]: pgmap v410: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T04:08:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:36 smithi196 ceph-mon[35731]: pgmap v410: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T04:08:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 -- 172.21.15.19:0/1281817969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc1025c0 msgr2=0x7f5acc1049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 --2- 172.21.15.19:0/1281817969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc1025c0 0x7f5acc1049b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009ca0 tx=0x7f5abc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 -- 172.21.15.19:0/1281817969 shutdown_connections 2024-09-19T04:08:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 --2- 172.21.15.19:0/1281817969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5acc107820 0x7f5acc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 --2- 172.21.15.19:0/1281817969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc104ef0 0x7f5acc1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 --2- 172.21.15.19:0/1281817969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc1025c0 0x7f5acc1049b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 -- 172.21.15.19:0/1281817969 >> 172.21.15.19:0/1281817969 conn(0x7f5acc0fc1e0 msgr2=0x7f5acc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 -- 172.21.15.19:0/1281817969 shutdown_connections 2024-09-19T04:08:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.474+0000 7f5ad087b640 1 -- 172.21.15.19:0/1281817969 wait complete. 2024-09-19T04:08:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ad087b640 1 Processor -- start 2024-09-19T04:08:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ad087b640 1 -- start start 2024-09-19T04:08:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ad087b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5acc1025c0 0x7f5acc197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ad087b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc104ef0 0x7f5acc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ad087b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc107820 0x7f5acc19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ad087b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc10d800 con 0x7f5acc107820 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ad087b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc10d680 con 0x7f5acc1025c0 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ad087b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc10d980 con 0x7f5acc104ef0 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5acad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc107820 0x7f5acc19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5acad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc107820 0x7f5acc19f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40498/0 (socket says 172.21.15.19:40498) 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5acad9d640 1 -- 172.21.15.19:0/4140340275 learned_addr learned my addr 172.21.15.19:0/4140340275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5aca59c640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5acc1025c0 0x7f5acc197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc104ef0 0x7f5acc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5acad9d640 1 -- 172.21.15.19:0/4140340275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc104ef0 msgr2=0x7f5acc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5acad9d640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc104ef0 0x7f5acc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5acad9d640 1 -- 172.21.15.19:0/4140340275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5acc1025c0 msgr2=0x7f5acc197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5acad9d640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5acc1025c0 0x7f5acc197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.475+0000 7f5acad9d640 1 -- 172.21.15.19:0/4140340275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc19fa80 con 0x7f5acc107820 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.476+0000 7f5acad9d640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc107820 0x7f5acc19f410 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400d720 tx=0x7f5ac400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.476+0000 7f5abb7fe640 1 -- 172.21.15.19:0/4140340275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4017070 con 0x7f5acc107820 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.476+0000 7f5abb7fe640 1 -- 172.21.15.19:0/4140340275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac400de30 con 0x7f5acc107820 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.476+0000 7f5abb7fe640 1 -- 172.21.15.19:0/4140340275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4004d90 con 0x7f5acc107820 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.476+0000 7f5ad087b640 1 -- 172.21.15.19:0/4140340275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc19fd70 con 0x7f5acc107820 2024-09-19T04:08:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.476+0000 7f5ad087b640 1 -- 172.21.15.19:0/4140340275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc1a0210 con 0x7f5acc107820 2024-09-19T04:08:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.479+0000 7f5abb7fe640 1 -- 172.21.15.19:0/4140340275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5ac4004f30 con 0x7f5acc107820 2024-09-19T04:08:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.479+0000 7f5ad3619740 1 Processor -- start 2024-09-19T04:08:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.479+0000 7f5ad3619740 1 -- start start 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e1fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 0x55bff2e27830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ad3619740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bff2d832f0 con 0x55bff2e1f9a0 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ad3619740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bff2d2b270 con 0x55bff2e202e0 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ad3619740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bff2d7f2b0 con 0x55bff2e25370 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ac9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e1fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ac9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e1fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40512/0 (socket says 172.21.15.19:40512) 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/734808075 learned_addr learned my addr 172.21.15.19:0/734808075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5abb7fe640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ab40a32f0 0x7f5ab40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5aca59c640 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 0x55bff2e27830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5acad9d640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ab40a32f0 0x7f5ab40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5abb7fe640 1 -- 172.21.15.19:0/4140340275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f5ac40cf550 con 0x7f5acc107820 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5aca59c640 1 -- 172.21.15.19:0/734808075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 msgr2=0x55bff2e27830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5aca59c640 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 0x55bff2e27830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5aca59c640 1 -- 172.21.15.19:0/734808075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 msgr2=0x55bff2e1fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5aca59c640 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e1fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.480+0000 7f5aca59c640 1 -- 172.21.15.19:0/734808075 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff2d2a340 con 0x55bff2e202e0 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.481+0000 7f5acad9d640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ab40a32f0 0x7f5ab40a57b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009610 tx=0x7f5ac4090d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.481+0000 7f5aca59c640 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e24da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5abc031ae0 tx=0x7f5abc005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859ad7285addfe26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.481+0000 7f5ab8ff9640 1 -- 172.21.15.19:0/734808075 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc049070 con 0x55bff2e202e0 2024-09-19T04:08:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.481+0000 7f5ab8ff9640 1 -- 172.21.15.19:0/734808075 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc036040 con 0x55bff2e202e0 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.481+0000 7f5ab8ff9640 1 -- 172.21.15.19:0/734808075 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc02fc10 con 0x55bff2e202e0 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- 172.21.15.19:0/734808075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 msgr2=0x55bff2e24da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e24da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5abc031ae0 tx=0x7f5abc005ef0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- 172.21.15.19:0/734808075 shutdown_connections 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 0x55bff2e27830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e24da0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 --2- 172.21.15.19:0/734808075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- 172.21.15.19:0/734808075 >> 172.21.15.19:0/734808075 conn(0x55bff2e1b010 msgr2=0x55bff2e1b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- 172.21.15.19:0/734808075 shutdown_connections 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- 172.21.15.19:0/734808075 wait complete. 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- start start 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 Processor -- start 2024-09-19T04:08:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 0x55bff2e299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 0x55bff2e29ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e30f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bff2e28160 con 0x55bff2e25370 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bff2d0c290 con 0x55bff2e1f9a0 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.482+0000 7f5ad3619740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bff2e282e0 con 0x55bff2e202e0 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ad3619740 10 client.0 _ll_register_callbacks cb 0x55bff2d3d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e30f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e30f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40516/0 (socket says 172.21.15.19:40516) 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/753764421 learned_addr learned my addr 172.21.15.19:0/753764421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5aca59c640 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 0x55bff2e29ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 0x55bff2e299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/753764421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 msgr2=0x55bff2e29ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 0x55bff2e29ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/753764421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 msgr2=0x55bff2e299b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 0x55bff2e299b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/753764421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff2a96210 con 0x55bff2e25370 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e30f30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0005fd0 tx=0x7f5ac000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9869d56636f7996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/753764421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0010430 con 0x55bff2e25370 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/753764421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f5ac00105d0 con 0x55bff2e25370 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ad3619740 1 -- 172.21.15.19:0/753764421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5abc036040 con 0x55bff2e25370 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.483+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/753764421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ac00107d0 con 0x55bff2e25370 2024-09-19T04:08:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ad3619740 10 client.15609 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ad3619740 1 -- 172.21.15.19:0/753764421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bff2b62620 con 0x55bff2e25370 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/753764421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc036040 con 0x55bff2e25370 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/753764421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5ac0021040 con 0x55bff2e25370 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ad3619740 10 client.15609 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ad3619740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ad3619740 20 client.15609 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ad3619740 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bff2e356a0 0x55bff2e37b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ad3619740 4 client.15609 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5ad3619740 1 -- 172.21.15.19:0/753764421 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bff2d90a20 con 0x55bff2e356a0 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.484+0000 7f5aca59c640 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bff2e356a0 0x55bff2e37b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.485+0000 7f5aca59c640 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bff2e356a0 0x55bff2e37b10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66792ba9205fcb35 server_cookie=f93c7bb87ecefe9f in_seq=0 out_seq=0 2024-09-19T04:08:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.485+0000 7f5ab1ffb640 10 client.15609 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.485+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/753764421 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bff2d90a20 con 0x55bff2e356a0 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.485+0000 7f5ab1ffb640 10 client.15609 handle_command_reply: tid=0 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 client.15609 shutdown 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 -- 172.21.15.19:0/753764421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 msgr2=0x55bff2e30f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e30f30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0005fd0 tx=0x7f5ac000b410 comp rx=0 tx=0).stop 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 -- 172.21.15.19:0/753764421 shutdown_connections 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e30f30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bff2e356a0 0x55bff2e37b10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 0x55bff2e29ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 --2- 172.21.15.19:0/753764421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 0x55bff2e299b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ab1ffb640 0 client.15609 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 -- 172.21.15.19:0/753764421 >> 172.21.15.19:0/753764421 conn(0x55bff2e1b010 msgr2=0x55bff2e1c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 -- 172.21.15.19:0/753764421 shutdown_connections 2024-09-19T04:08:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 1 -- 172.21.15.19:0/753764421 wait complete. 2024-09-19T04:08:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.486+0000 7f5ad3619740 20 client.15609 trim_cache size 0 max 16384 2024-09-19T04:08:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.500+0000 7f5ad3619740 1 Processor -- start 2024-09-19T04:08:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.500+0000 7f5ad3619740 1 -- start start 2024-09-19T04:08:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.500+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 0x55bff2e28dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.500+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 0x55bff2e30ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e1bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ad3619740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1aa680 con 0x55bff2e25370 2024-09-19T04:08:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ad3619740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1ac1c0 con 0x55bff2e1f9a0 2024-09-19T04:08:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ad3619740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1aad50 con 0x55bff2e202e0 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e1bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e1bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40518/0 (socket says 172.21.15.19:40518) 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/1455235915 learned_addr learned my addr 172.21.15.19:0/1455235915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 0x55bff2e28dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5aca59c640 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 0x55bff2e30ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/1455235915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 msgr2=0x55bff2e30ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 0x55bff2e30ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/1455235915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 msgr2=0x55bff2e28dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 0x55bff2e28dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/1455235915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff2e162d0 con 0x55bff2e25370 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e1bec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000ec70 tx=0x7f5ac0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e67862f23982d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ab3fff640 1 -- 172.21.15.19:0/1455235915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0010430 con 0x55bff2e25370 2024-09-19T04:08:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ab3fff640 1 -- 172.21.15.19:0/1455235915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac00105d0 con 0x55bff2e25370 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.501+0000 7f5ab3fff640 1 -- 172.21.15.19:0/1455235915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0012290 con 0x55bff2e25370 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 -- 172.21.15.19:0/1455235915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 msgr2=0x55bff2e1bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e1bec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000ec70 tx=0x7f5ac0001e50 comp rx=0 tx=0).stop 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 -- 172.21.15.19:0/1455235915 shutdown_connections 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e25370 0x55bff2e1bec0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e202e0 0x55bff2e30ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1455235915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e1f9a0 0x55bff2e28dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 -- 172.21.15.19:0/1455235915 >> 172.21.15.19:0/1455235915 conn(0x55bff2e1b010 msgr2=0x55bff2e286b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 -- 172.21.15.19:0/1455235915 shutdown_connections 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 -- 172.21.15.19:0/1455235915 wait complete. 2024-09-19T04:08:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 -- start start 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.502+0000 7f5ad3619740 1 Processor -- start 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e2e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e2e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ad3619740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 0x55bff2e2f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ad3619740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1abaf0 con 0x55bff2e1f9a0 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ad3619740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1ab420 con 0x55bff2e202e0 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ad3619740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1a8660 con 0x55bff2e25370 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ad3619740 10 client.0 _ll_register_callbacks cb 0x7f5ac0012b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e2e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e2e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40526/0 (socket says 172.21.15.19:40526) 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/1862524957 learned_addr learned my addr 172.21.15.19:0/1862524957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5aca59c640 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e2e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/1862524957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 msgr2=0x55bff2e2f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 0x55bff2e2f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 msgr2=0x55bff2e2e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e2e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 -- 172.21.15.19:0/1862524957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff2e29300 con 0x55bff2e1f9a0 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5aca59c640 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e2e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ac9d9b640 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e2e490 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00234f0 tx=0x7f5ac00239a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7768784afdb069 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/1862524957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0039030 con 0x55bff2e1f9a0 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.503+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/1862524957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f5ac00236d0 con 0x55bff2e1f9a0 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 1 -- 172.21.15.19:0/1862524957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac00105d0 con 0x55bff2e1f9a0 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/1862524957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ac0010430 con 0x55bff2e1f9a0 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 10 client.15615 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 1 -- 172.21.15.19:0/1862524957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bff2e310e0 con 0x55bff2e1f9a0 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/1862524957 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac00105d0 con 0x55bff2e1f9a0 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/1862524957 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5ac0010a00 con 0x55bff2e1f9a0 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 10 client.15615 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 20 client.15615 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bff2ea08e0 0x55bff2ea2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 4 client.15615 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:08:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5ad3619740 1 -- 172.21.15.19:0/1862524957 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bff2d8ad70 con 0x55bff2ea08e0 2024-09-19T04:08:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.504+0000 7f5aca59c640 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bff2ea08e0 0x55bff2ea2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.505+0000 7f5aca59c640 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bff2ea08e0 0x55bff2ea2d00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db13aab6e40e6f1b server_cookie=188d8de608881152 in_seq=0 out_seq=0 2024-09-19T04:08:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.505+0000 7f5ab1ffb640 10 client.15615 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.514+0000 7f5ab1ffb640 1 -- 172.21.15.19:0/1862524957 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bff2d8ad70 con 0x55bff2ea08e0 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.514+0000 7f5ab1ffb640 10 client.15615 handle_command_reply: tid=0 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.514+0000 7f5ad3619740 1 client.15615 shutdown 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.514+0000 7f5ad3619740 1 -- 172.21.15.19:0/1862524957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 msgr2=0x55bff2e2e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.514+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e2e490 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00234f0 tx=0x7f5ac00239a0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 1 -- 172.21.15.19:0/1862524957 shutdown_connections 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55bff2ea08e0 0x55bff2ea2d00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bff2e25370 0x55bff2e2f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ab1ffb640 0 client.15615 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bff2e202e0 0x55bff2e2e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 1 --2- 172.21.15.19:0/1862524957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bff2e1f9a0 0x55bff2e2e490 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 1 -- 172.21.15.19:0/1862524957 >> 172.21.15.19:0/1862524957 conn(0x55bff2e1b010 msgr2=0x55bff2e199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 1 -- 172.21.15.19:0/1862524957 shutdown_connections 2024-09-19T04:08:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 1 -- 172.21.15.19:0/1862524957 wait complete. 2024-09-19T04:08:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad3619740 20 client.15615 trim_cache size 0 max 16384 2024-09-19T04:08:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:08:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:08:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:08:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:08:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 -- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ab40a32f0 msgr2=0x7f5ab40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ab40a32f0 0x7f5ab40a57b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009610 tx=0x7f5ac4090d00 comp rx=0 tx=0).stop 2024-09-19T04:08:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 -- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc107820 msgr2=0x7f5acc19f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc107820 0x7f5acc19f410 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400d720 tx=0x7f5ac400dbe0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 -- 172.21.15.19:0/4140340275 shutdown_connections 2024-09-19T04:08:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f5ab40a32f0 0x7f5ab40a57b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5acc107820 0x7f5acc19f410 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc104ef0 0x7f5acc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 --2- 172.21.15.19:0/4140340275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5acc1025c0 0x7f5acc197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 -- 172.21.15.19:0/4140340275 >> 172.21.15.19:0/4140340275 conn(0x7f5acc0fc1e0 msgr2=0x7f5acc102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.515+0000 7f5ad087b640 1 -- 172.21.15.19:0/4140340275 shutdown_connections 2024-09-19T04:08:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:36.516+0000 7f5ad087b640 1 -- 172.21.15.19:0/4140340275 wait complete. 2024-09-19T04:08:37.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:08:37.315 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:08:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1943936485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 msgr2=0x7f1a0c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1943936485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 0x7f1a0c103170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009930 tx=0x7f19fc02a420 comp rx=0 tx=0).stop 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1943936485 shutdown_connections 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1943936485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a0c108290 0x7f1a0c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1943936485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a0c103740 0x7f1a0c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1943936485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 0x7f1a0c103170 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1943936485 >> 172.21.15.19:0/1943936485 conn(0x7f1a0c0fc7d0 msgr2=0x7f1a0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1943936485 shutdown_connections 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.807+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1943936485 wait complete. 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a11ac9640 1 Processor -- start 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a11ac9640 1 -- start start 2024-09-19T04:08:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a11ac9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 0x7f1a0c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a11ac9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a0c103740 0x7f1a0c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a11ac9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a0c108290 0x7f1a0c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a11ac9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c112150 con 0x7f1a0c102d90 2024-09-19T04:08:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a11ac9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c111fd0 con 0x7f1a0c103740 2024-09-19T04:08:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a11ac9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c1122d0 con 0x7f1a0c108290 2024-09-19T04:08:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a0b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 0x7f1a0c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a0b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 0x7f1a0c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40548/0 (socket says 172.21.15.19:40548) 2024-09-19T04:08:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.808+0000 7f1a0b7fe640 1 -- 172.21.15.19:0/1066506673 learned_addr learned my addr 172.21.15.19:0/1066506673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a0b7fe640 1 -- 172.21.15.19:0/1066506673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a0c108290 msgr2=0x7f1a0c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a0bfff640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a0c108290 0x7f1a0c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a03fff640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a0c103740 0x7f1a0c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a0b7fe640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a0c108290 0x7f1a0c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a0b7fe640 1 -- 172.21.15.19:0/1066506673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a0c103740 msgr2=0x7f1a0c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a0b7fe640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a0c103740 0x7f1a0c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a0b7fe640 1 -- 172.21.15.19:0/1066506673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c19d6c0 con 0x7f1a0c102d90 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a0bfff640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a0c108290 0x7f1a0c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:08:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a03fff640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a0c103740 0x7f1a0c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:08:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a0b7fe640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 0x7f1a0c198710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009a50 tx=0x7f19fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a097fa640 1 -- 172.21.15.19:0/1066506673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc03f030 con 0x7f1a0c102d90 2024-09-19T04:08:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c19d950 con 0x7f1a0c102d90 2024-09-19T04:08:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a097fa640 1 -- 172.21.15.19:0/1066506673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19fc003ce0 con 0x7f1a0c102d90 2024-09-19T04:08:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a097fa640 1 -- 172.21.15.19:0/1066506673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc035070 con 0x7f1a0c102d90 2024-09-19T04:08:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.809+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c1a5290 con 0x7f1a0c102d90 2024-09-19T04:08:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.810+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0c06a7f0 con 0x7f1a0c102d90 2024-09-19T04:08:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.811+0000 7f1a097fa640 1 -- 172.21.15.19:0/1066506673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f19fc042030 con 0x7f1a0c102d90 2024-09-19T04:08:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.812+0000 7f1a097fa640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19f80a3240 0x7f19f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.812+0000 7f1a097fa640 1 -- 172.21.15.19:0/1066506673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f19fc0f0800 con 0x7f1a0c102d90 2024-09-19T04:08:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.812+0000 7f1a03fff640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19f80a3240 0x7f19f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.813+0000 7f1a03fff640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19f80a3240 0x7f19f80a5700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006fd0 tx=0x7f19f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:37.815+0000 7f1a097fa640 1 -- 172.21.15.19:0/1066506673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f19fc0bd120 con 0x7f1a0c102d90 2024-09-19T04:08:38.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.029+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a0c10a2a0 con 0x7f1a0c102d90 2024-09-19T04:08:38.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.030+0000 7f1a097fa640 1 -- 172.21.15.19:0/1066506673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7f19fc0bca20 con 0x7f1a0c102d90 2024-09-19T04:08:38.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:08:38.033 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:08:38.033 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:08:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19f80a3240 msgr2=0x7f19f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19f80a3240 0x7f19f80a5700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006fd0 tx=0x7f19f4008040 comp rx=0 tx=0).stop 2024-09-19T04:08:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 msgr2=0x7f1a0c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 0x7f1a0c198710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009a50 tx=0x7f19fc004270 comp rx=0 tx=0).stop 2024-09-19T04:08:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 shutdown_connections 2024-09-19T04:08:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f19f80a3240 0x7f19f80a5700 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a0c108290 0x7f1a0c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a0c103740 0x7f1a0c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 --2- 172.21.15.19:0/1066506673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c102d90 0x7f1a0c198710 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:38.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.032+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 >> 172.21.15.19:0/1066506673 conn(0x7f1a0c0fc7d0 msgr2=0x7f1a0c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:38.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.033+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 shutdown_connections 2024-09-19T04:08:38.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:38.033+0000 7f1a11ac9640 1 -- 172.21.15.19:0/1066506673 wait complete. 2024-09-19T04:08:38.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:38 smithi019 ceph-mon[30938]: pgmap v411: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-19T04:08:38.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:38 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1066506673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:38.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:38 smithi111 ceph-mon[35365]: pgmap v411: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-19T04:08:38.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:38 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1066506673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:38.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:38 smithi196 ceph-mon[35731]: pgmap v411: 129 pgs: 129 active+clean; 712 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-19T04:08:38.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:38 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1066506673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:38.767 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 75de194c --release --await' 2024-09-19T04:08:38.768 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 75de194c --release --await 2024-09-19T04:08:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 -- 172.21.15.19:0/2639569144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4102790 msgr2=0x7efcb41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 --2- 172.21.15.19:0/2639569144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4102790 0x7efcb41067e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009900 tx=0x7efc9c02a410 comp rx=0 tx=0).stop 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 -- 172.21.15.19:0/2639569144 shutdown_connections 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 --2- 172.21.15.19:0/2639569144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcb4106f10 0x7efcb410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 --2- 172.21.15.19:0/2639569144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4102790 0x7efcb41067e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 --2- 172.21.15.19:0/2639569144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb4101de0 0x7efcb41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 -- 172.21.15.19:0/2639569144 >> 172.21.15.19:0/2639569144 conn(0x7efcb40fd8c0 msgr2=0x7efcb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 -- 172.21.15.19:0/2639569144 shutdown_connections 2024-09-19T04:08:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.483+0000 7efcba8e9640 1 -- 172.21.15.19:0/2639569144 wait complete. 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcba8e9640 1 Processor -- start 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcba8e9640 1 -- start start 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcba8e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4101de0 0x7efcb41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcba8e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb4102790 0x7efcb41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcba8e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcb4106f10 0x7efcb419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcba8e9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4111fb0 con 0x7efcb4101de0 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcba8e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4111e30 con 0x7efcb4106f10 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcba8e9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4112130 con 0x7efcb4102790 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4101de0 0x7efcb41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4101de0 0x7efcb41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40568/0 (socket says 172.21.15.19:40568) 2024-09-19T04:08:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.484+0000 7efcb3fff640 1 -- 172.21.15.19:0/748490319 learned_addr learned my addr 172.21.15.19:0/748490319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb8e86640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcb4106f10 0x7efcb419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcabfff640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb4102790 0x7efcb41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb3fff640 1 -- 172.21.15.19:0/748490319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb4102790 msgr2=0x7efcb41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb3fff640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb4102790 0x7efcb41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb3fff640 1 -- 172.21.15.19:0/748490319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcb4106f10 msgr2=0x7efcb419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb3fff640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcb4106f10 0x7efcb419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb3fff640 1 -- 172.21.15.19:0/748490319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb419d460 con 0x7efcb4101de0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb3fff640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4101de0 0x7efcb41984b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efca400c970 tx=0x7efca400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb1ffb640 1 -- 172.21.15.19:0/748490319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4007bb0 con 0x7efcb4101de0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb1ffb640 1 -- 172.21.15.19:0/748490319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca4007d50 con 0x7efcb4101de0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcba8e9640 1 -- 172.21.15.19:0/748490319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb419d750 con 0x7efcb4101de0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcb1ffb640 1 -- 172.21.15.19:0/748490319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca400f450 con 0x7efcb4101de0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.485+0000 7efcba8e9640 1 -- 172.21.15.19:0/748490319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a5030 con 0x7efcb4101de0 2024-09-19T04:08:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.488+0000 7efcb1ffb640 1 -- 172.21.15.19:0/748490319 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7efca4005020 con 0x7efcb4101de0 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.488+0000 7efcbccc2740 1 Processor -- start 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.488+0000 7efcbccc2740 1 -- start start 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.488+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf827ff20 0x557bf8280300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf828f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf8291a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcbccc2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bf81e5620 con 0x557bf8280840 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcbccc2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557bf818ce50 con 0x557bf827ff20 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcbccc2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bf81e15e0 con 0x557bf828f650 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf828f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf828f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40584/0 (socket says 172.21.15.19:40584) 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 -- 172.21.15.19:0/2719726686 learned_addr learned my addr 172.21.15.19:0/2719726686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb8e86640 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf827ff20 0x557bf8280300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcabfff640 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf8291a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 -- 172.21.15.19:0/2719726686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 msgr2=0x557bf8291a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf8291a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 -- 172.21.15.19:0/2719726686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf827ff20 msgr2=0x557bf8280300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf827ff20 0x557bf8280300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 -- 172.21.15.19:0/2719726686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf817dd40 con 0x557bf8280840 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcabfff640 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf8291a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb1ffb640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efca00a32c0 0x7efca00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb8e86640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efca00a32c0 0x7efca00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb3fff640 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf828f110 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efca40d3c60 tx=0x7efca40d8590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96802d9398620d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.489+0000 7efcb1ffb640 1 -- 172.21.15.19:0/748490319 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7efca40cfac0 con 0x7efcb4101de0 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcaa7fc640 1 -- 172.21.15.19:0/2719726686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca401d070 con 0x557bf8280840 2024-09-19T04:08:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcaa7fc640 1 -- 172.21.15.19:0/2719726686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca409c440 con 0x557bf8280840 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcaa7fc640 1 -- 172.21.15.19:0/2719726686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca409b880 con 0x557bf8280840 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 -- 172.21.15.19:0/2719726686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 msgr2=0x557bf828f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcb8e86640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efca00a32c0 0x7efca00a5780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efcac00b4d0 tx=0x7efcac00e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf828f110 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efca40d3c60 tx=0x7efca40d8590 comp rx=0 tx=0).stop 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 -- 172.21.15.19:0/2719726686 shutdown_connections 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf8291a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf828f110 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 --2- 172.21.15.19:0/2719726686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf827ff20 0x557bf8280300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 -- 172.21.15.19:0/2719726686 >> 172.21.15.19:0/2719726686 conn(0x557bf827c5f0 msgr2=0x557bf827c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 -- 172.21.15.19:0/2719726686 shutdown_connections 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.490+0000 7efcbccc2740 1 -- 172.21.15.19:0/2719726686 wait complete. 2024-09-19T04:08:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 1 -- start start 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 1 Processor -- start 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf828c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf8280840 0x557bf828caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf828cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bf8292220 con 0x557bf827ff20 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557bf816df10 con 0x557bf8280840 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bf82923a0 con 0x557bf828f650 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcbccc2740 10 client.0 _ll_register_callbacks cb 0x557bf8191db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf828c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf828c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40590/0 (socket says 172.21.15.19:40590) 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.491+0000 7efcb3fff640 1 -- 172.21.15.19:0/695415412 learned_addr learned my addr 172.21.15.19:0/695415412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcabfff640 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf8280840 0x557bf828caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcb8e86640 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf828cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcb3fff640 1 -- 172.21.15.19:0/695415412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 msgr2=0x557bf828cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcb3fff640 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf828cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcb3fff640 1 -- 172.21.15.19:0/695415412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf8280840 msgr2=0x557bf828caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcb3fff640 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf8280840 0x557bf828caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcb3fff640 1 -- 172.21.15.19:0/695415412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf7ef4580 con 0x557bf827ff20 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcb8e86640 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf828cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcb3fff640 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf828c560 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efca40cd590 tx=0x7efca4004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d05f683b0df900d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efc937fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efc937fe640 1 -- 172.21.15.19:0/695415412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca40b9060 con 0x557bf827ff20 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efc937fe640 1 -- 172.21.15.19:0/695415412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7efca409a040 con 0x557bf827ff20 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcbccc2740 1 -- 172.21.15.19:0/695415412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca409c440 con 0x557bf827ff20 2024-09-19T04:08:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efc937fe640 1 -- 172.21.15.19:0/695415412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efca401d040 con 0x557bf827ff20 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcbccc2740 10 client.15633 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.492+0000 7efcbccc2740 1 -- 172.21.15.19:0/695415412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557bf7fbe960 con 0x557bf827ff20 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efc937fe640 1 -- 172.21.15.19:0/695415412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca409c440 con 0x557bf827ff20 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efc937fe640 1 -- 172.21.15.19:0/695415412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7efca4005310 con 0x557bf827ff20 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efcbccc2740 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efcbccc2740 10 client.15633 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efcbccc2740 20 client.15633 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efcbccc2740 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x557bf829f0c0 0x557bf82a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efcbccc2740 4 client.15633 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efcbccc2740 1 -- 172.21.15.19:0/695415412 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bf81f2b90 con 0x557bf829f0c0 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.493+0000 7efcabfff640 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x557bf829f0c0 0x557bf82a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.494+0000 7efcabfff640 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x557bf829f0c0 0x557bf82a14b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb03bf2f19edb151 server_cookie=188511522e549427 in_seq=0 out_seq=0 2024-09-19T04:08:39.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.494+0000 7efc937fe640 10 client.15633 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:39.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.494+0000 7efc937fe640 1 -- 172.21.15.19:0/695415412 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557bf81f2b90 con 0x557bf829f0c0 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efc937fe640 10 client.15633 handle_command_reply: tid=0 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 client.15633 shutdown 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 -- 172.21.15.19:0/695415412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 msgr2=0x557bf828c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf828c560 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efca40cd590 tx=0x7efca4004990 comp rx=0 tx=0).stop 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 -- 172.21.15.19:0/695415412 shutdown_connections 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x557bf829f0c0 0x557bf82a14b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf828f650 0x557bf828cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf8280840 0x557bf828caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 --2- 172.21.15.19:0/695415412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf828c560 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 -- 172.21.15.19:0/695415412 >> 172.21.15.19:0/695415412 conn(0x557bf827c5f0 msgr2=0x557bf8282350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efc937fe640 0 client.15633 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 -- 172.21.15.19:0/695415412 shutdown_connections 2024-09-19T04:08:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 1 -- 172.21.15.19:0/695415412 wait complete. 2024-09-19T04:08:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.495+0000 7efcbccc2740 20 client.15633 trim_cache size 0 max 16384 2024-09-19T04:08:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.512+0000 7efcbccc2740 1 Processor -- start 2024-09-19T04:08:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.512+0000 7efcbccc2740 1 -- start start 2024-09-19T04:08:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.512+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf827ff20 0x557bf829ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.512+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf82897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.512+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8289d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.512+0000 7efcbccc2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41acb50 con 0x557bf8280840 2024-09-19T04:08:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.512+0000 7efcbccc2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ae690 con 0x557bf828f650 2024-09-19T04:08:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.512+0000 7efcbccc2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ad220 con 0x557bf827ff20 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf82897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf82897c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40600/0 (socket says 172.21.15.19:40600) 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 -- 172.21.15.19:0/1906103382 learned_addr learned my addr 172.21.15.19:0/1906103382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcb8e86640 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8289d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcb3fff640 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf827ff20 0x557bf829ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 -- 172.21.15.19:0/1906103382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf827ff20 msgr2=0x557bf829ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf827ff20 0x557bf829ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 -- 172.21.15.19:0/1906103382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 msgr2=0x557bf8289d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8289d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 -- 172.21.15.19:0/1906103382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf8291de0 con 0x557bf8280840 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcb8e86640 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8289d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efcabfff640 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf82897c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc9c031550 tx=0x7efc9c0319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36b599b2be4ec9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:39.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efca9ffb640 1 -- 172.21.15.19:0/1906103382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c035040 con 0x557bf8280840 2024-09-19T04:08:39.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efca9ffb640 1 -- 172.21.15.19:0/1906103382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc9c005500 con 0x557bf8280840 2024-09-19T04:08:39.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.513+0000 7efca9ffb640 1 -- 172.21.15.19:0/1906103382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c004e00 con 0x557bf8280840 2024-09-19T04:08:39.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 -- 172.21.15.19:0/1906103382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 msgr2=0x557bf82897c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf82897c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc9c031550 tx=0x7efc9c0319d0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 -- 172.21.15.19:0/1906103382 shutdown_connections 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8289d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf8280840 0x557bf82897c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 --2- 172.21.15.19:0/1906103382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf827ff20 0x557bf829ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 -- 172.21.15.19:0/1906103382 >> 172.21.15.19:0/1906103382 conn(0x557bf827c5f0 msgr2=0x557bf829a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 -- 172.21.15.19:0/1906103382 shutdown_connections 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 -- 172.21.15.19:0/1906103382 wait complete. 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 -- start start 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.514+0000 7efcbccc2740 1 Processor -- start 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf82a2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf8280840 0x557bf82a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcbccc2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8308a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcbccc2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41adfc0 con 0x557bf827ff20 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcbccc2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ad8f0 con 0x557bf828f650 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcbccc2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41aab30 con 0x557bf8280840 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcbccc2740 10 client.0 _ll_register_callbacks cb 0x7efca40980a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf82a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf82a2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40614/0 (socket says 172.21.15.19:40614) 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 -- 172.21.15.19:0/3397194272 learned_addr learned my addr 172.21.15.19:0/3397194272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcabfff640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf8280840 0x557bf82a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb8e86640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8308a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 -- 172.21.15.19:0/3397194272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf8280840 msgr2=0x557bf82a2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf8280840 0x557bf82a2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 -- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 msgr2=0x557bf8308a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8308a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 -- 172.21.15.19:0/3397194272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf8309040 con 0x557bf827ff20 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcabfff640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf8280840 0x557bf82a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb8e86640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8308a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efcb3fff640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf82a2690 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efca40986c0 tx=0x7efca40cfdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe2f6da53f0eec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.515+0000 7efc937fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efc937fe640 1 -- 172.21.15.19:0/3397194272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca40b9040 con 0x557bf827ff20 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efc937fe640 1 -- 172.21.15.19:0/3397194272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7efca40a4720 con 0x557bf827ff20 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 1 -- 172.21.15.19:0/3397194272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc9c005500 con 0x557bf827ff20 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efc937fe640 1 -- 172.21.15.19:0/3397194272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efca40ab050 con 0x557bf827ff20 2024-09-19T04:08:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 10 client.15639 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 1 -- 172.21.15.19:0/3397194272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557bf8309040 con 0x557bf827ff20 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efc937fe640 1 -- 172.21.15.19:0/3397194272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc9c005500 con 0x557bf827ff20 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efc937fe640 1 -- 172.21.15.19:0/3397194272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7efca4083020 con 0x557bf827ff20 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 10 client.15639 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 10 client.15639 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 20 client.15639 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x557bf830a6c0 0x557bf830cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 4 client.15639 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "75de194c", "release": true, "await": true}] 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.516+0000 7efcbccc2740 1 -- 172.21.15.19:0/3397194272 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "quiesce db", "set_id": "75de194c", "release": true, "await": true}) -- 0x557bf81ed0a0 con 0x557bf830a6c0 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.517+0000 7efcabfff640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x557bf830a6c0 0x557bf830cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:39.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.517+0000 7efcabfff640 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x557bf830a6c0 0x557bf830cab0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9259cf21c2f6a77e server_cookie=c54b6fae21f7ad29 in_seq=0 out_seq=0 2024-09-19T04:08:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.517+0000 7efc937fe640 10 client.15639 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efc937fe640 1 -- 172.21.15.19:0/3397194272 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x557bf81ed0a0 con 0x557bf830a6c0 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efc937fe640 10 client.15639 handle_command_reply: tid=0 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 client.15639 shutdown 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 -- 172.21.15.19:0/3397194272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 msgr2=0x557bf82a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf82a2690 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efca40986c0 tx=0x7efca40cfdc0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 -- 172.21.15.19:0/3397194272 shutdown_connections 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557bf828f650 0x557bf8308a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x557bf830a6c0 0x557bf830cab0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf8280840 0x557bf82a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 --2- 172.21.15.19:0/3397194272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf827ff20 0x557bf82a2690 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 -- 172.21.15.19:0/3397194272 >> 172.21.15.19:0/3397194272 conn(0x557bf827c5f0 msgr2=0x557bf827b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efc937fe640 0 client.15639 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 -- 172.21.15.19:0/3397194272 shutdown_connections 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 1 -- 172.21.15.19:0/3397194272 wait complete. 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.521+0000 7efcbccc2740 20 client.15639 trim_cache size 0 max 16384 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 29, 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24476, 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 8, 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-19T04:08:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "75de194c": { 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 8, 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.3, 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-19T04:08:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:08:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-19T04:08:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:08:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:39.528 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 -- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efca00a32c0 msgr2=0x7efca00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efca00a32c0 0x7efca00a5780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efcac00b4d0 tx=0x7efcac00e040 comp rx=0 tx=0).stop 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 -- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4101de0 msgr2=0x7efcb41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4101de0 0x7efcb41984b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efca400c970 tx=0x7efca400ce30 comp rx=0 tx=0).stop 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 -- 172.21.15.19:0/748490319 shutdown_connections 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7efca00a32c0 0x7efca00a5780 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcb4106f10 0x7efcb419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb4102790 0x7efcb41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 --2- 172.21.15.19:0/748490319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb4101de0 0x7efcb41984b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 -- 172.21.15.19:0/748490319 >> 172.21.15.19:0/748490319 conn(0x7efcb40fd8c0 msgr2=0x7efcb40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 -- 172.21.15.19:0/748490319 shutdown_connections 2024-09-19T04:08:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:39.522+0000 7efcba8e9640 1 -- 172.21.15.19:0/748490319 wait complete. 2024-09-19T04:08:40.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:40 smithi019 ceph-mon[30938]: pgmap v412: 129 pgs: 129 active+clean; 717 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-09-19T04:08:40.243 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '75de194c', total seconds elapsed: 23.6 2024-09-19T04:08:40.243 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T04:08:40.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:08:40.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:40 smithi111 ceph-mon[35365]: pgmap v412: 129 pgs: 129 active+clean; 717 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-09-19T04:08:40.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:40 smithi196 ceph-mon[35731]: pgmap v412: 129 pgs: 129 active+clean; 717 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-09-19T04:08:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:08:40.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 -- 172.21.15.19:0/321491914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18101400 msgr2=0x7f4e1810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/321491914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18101400 0x7f4e1810d930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009930 tx=0x7f4e1002a420 comp rx=0 tx=0).stop 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 -- 172.21.15.19:0/321491914 shutdown_connections 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/321491914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18101400 0x7f4e1810d930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/321491914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e18100a60 0x7f4e18100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/321491914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e18106a60 0x7f4e18106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 -- 172.21.15.19:0/321491914 >> 172.21.15.19:0/321491914 conn(0x7f4e180fc780 msgr2=0x7f4e180feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 -- 172.21.15.19:0/321491914 shutdown_connections 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.970+0000 7f4e1ee12640 1 -- 172.21.15.19:0/321491914 wait complete. 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.971+0000 7f4e1ee12640 1 Processor -- start 2024-09-19T04:08:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.971+0000 7f4e1ee12640 1 -- start start 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1ee12640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e18100a60 0x7f4e181992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1ee12640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e18101400 0x7f4e18199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1ee12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18106a60 0x7f4e181a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1ee12640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18110e80 con 0x7f4e18106a60 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1ee12640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18110d00 con 0x7f4e18101400 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1ee12640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18111000 con 0x7f4e18100a60 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1d3af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18106a60 0x7f4e181a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e18100a60 0x7f4e181992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e18100a60 0x7f4e181992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:34928/0 (socket says 172.21.15.19:34928) 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 -- 172.21.15.19:0/728777466 learned_addr learned my addr 172.21.15.19:0/728777466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1d3af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18106a60 0x7f4e181a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40622/0 (socket says 172.21.15.19:40622) 2024-09-19T04:08:40.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e17fff640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e18101400 0x7f4e18199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 -- 172.21.15.19:0/728777466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e18101400 msgr2=0x7f4e18199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e18101400 0x7f4e18199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 -- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18106a60 msgr2=0x7f4e181a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18106a60 0x7f4e181a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 -- 172.21.15.19:0/728777466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e181a4890 con 0x7f4e18100a60 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1d3af640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18106a60 0x7f4e181a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e1cbae640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e18100a60 0x7f4e181992d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800ba50 tx=0x7f4e0800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.972+0000 7f4e15ffb640 1 -- 172.21.15.19:0/728777466 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08019070 con 0x7f4e18100a60 2024-09-19T04:08:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.973+0000 7f4e15ffb640 1 -- 172.21.15.19:0/728777466 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e0800cbe0 con 0x7f4e18100a60 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.973+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e181a4a60 con 0x7f4e18100a60 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.973+0000 7f4e15ffb640 1 -- 172.21.15.19:0/728777466 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e080076e0 con 0x7f4e18100a60 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.973+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e181a4f70 con 0x7f4e18100a60 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.974+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e18100ec0 con 0x7f4e18100a60 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.975+0000 7f4e15ffb640 1 -- 172.21.15.19:0/728777466 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4e0801c070 con 0x7f4e18100a60 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.976+0000 7f4e15ffb640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e040a3240 0x7f4e040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.976+0000 7f4e17fff640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e040a3240 0x7f4e040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.976+0000 7f4e15ffb640 1 -- 172.21.15.19:0/728777466 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f4e080cf420 con 0x7f4e18100a60 2024-09-19T04:08:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.976+0000 7f4e17fff640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e040a3240 0x7f4e040a5700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e1806b2e0 tx=0x7f4e00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:40.980+0000 7f4e15ffb640 1 -- 172.21.15.19:0/728777466 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e0809b6f0 con 0x7f4e18100a60 2024-09-19T04:08:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.206+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e1806ad80 con 0x7f4e18100a60 2024-09-19T04:08:41.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.207+0000 7f4e15ffb640 1 -- 172.21.15.19:0/728777466 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7f4e1806ad80 con 0x7f4e18100a60 2024-09-19T04:08:41.208 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:08:41.209 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:08:41.209 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:08:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e040a3240 msgr2=0x7f4e040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e040a3240 0x7f4e040a5700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e1806b2e0 tx=0x7f4e00005f90 comp rx=0 tx=0).stop 2024-09-19T04:08:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e18100a60 msgr2=0x7f4e181992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e18100a60 0x7f4e181992d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800ba50 tx=0x7f4e0800bf10 comp rx=0 tx=0).stop 2024-09-19T04:08:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 shutdown_connections 2024-09-19T04:08:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f4e040a3240 0x7f4e040a5700 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e18106a60 0x7f4e181a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e18101400 0x7f4e18199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 --2- 172.21.15.19:0/728777466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e18100a60 0x7f4e181992d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 >> 172.21.15.19:0/728777466 conn(0x7f4e180fc780 msgr2=0x7f4e180fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 shutdown_connections 2024-09-19T04:08:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:41.209+0000 7f4e1ee12640 1 -- 172.21.15.19:0/728777466 wait complete. 2024-09-19T04:08:41.886 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T04:08:41.886 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T04:08:42.154 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:08:42.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:42 smithi111 ceph-mon[35365]: pgmap v413: 129 pgs: 129 active+clean; 717 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T04:08:42.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:42 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/728777466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:42 smithi019 ceph-mon[30938]: pgmap v413: 129 pgs: 129 active+clean; 717 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T04:08:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:42 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/728777466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:42.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:42 smithi196 ceph-mon[35731]: pgmap v413: 129 pgs: 129 active+clean; 717 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T04:08:42.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:42 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/728777466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:08:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 -- 172.21.15.19:0/3054226715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c102ef0 msgr2=0x7eff1c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 --2- 172.21.15.19:0/3054226715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c102ef0 0x7eff1c10da40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7eff14009ca0 tx=0x7eff1402a950 comp rx=0 tx=0).stop 2024-09-19T04:08:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 -- 172.21.15.19:0/3054226715 shutdown_connections 2024-09-19T04:08:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 --2- 172.21.15.19:0/3054226715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c102ef0 0x7eff1c10da40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 --2- 172.21.15.19:0/3054226715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff1c102570 0x7eff1c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 --2- 172.21.15.19:0/3054226715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff1c100e20 0x7eff1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 -- 172.21.15.19:0/3054226715 >> 172.21.15.19:0/3054226715 conn(0x7eff1c0fc7d0 msgr2=0x7eff1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 -- 172.21.15.19:0/3054226715 shutdown_connections 2024-09-19T04:08:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.665+0000 7eff24758640 1 -- 172.21.15.19:0/3054226715 wait complete. 2024-09-19T04:08:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff24758640 1 Processor -- start 2024-09-19T04:08:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff24758640 1 -- start start 2024-09-19T04:08:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff24758640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c100e20 0x7eff1c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff24758640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff1c102570 0x7eff1c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff24758640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff1c102ef0 0x7eff1c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff24758640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c110fc0 con 0x7eff1c100e20 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff24758640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c110e40 con 0x7eff1c102ef0 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff24758640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c111140 con 0x7eff1c102570 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff224f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c100e20 0x7eff1c197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff21cf3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff1c102570 0x7eff1c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff224f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c100e20 0x7eff1c197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55012/0 (socket says 172.21.15.19:55012) 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff224f4640 1 -- 172.21.15.19:0/2146217388 learned_addr learned my addr 172.21.15.19:0/2146217388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.666+0000 7eff22cf5640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff1c102ef0 0x7eff1c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff21cf3640 1 -- 172.21.15.19:0/2146217388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff1c102ef0 msgr2=0x7eff1c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff21cf3640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff1c102ef0 0x7eff1c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff21cf3640 1 -- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c100e20 msgr2=0x7eff1c197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff21cf3640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c100e20 0x7eff1c197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff21cf3640 1 -- 172.21.15.19:0/2146217388 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c1a69b0 con 0x7eff1c102570 2024-09-19T04:08:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff22cf5640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff1c102ef0 0x7eff1c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:08:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff21cf3640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff1c102570 0x7eff1c197680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00e9f0 tx=0x7eff0c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff137fe640 1 -- 172.21.15.19:0/2146217388 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00ccc0 con 0x7eff1c102570 2024-09-19T04:08:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff137fe640 1 -- 172.21.15.19:0/2146217388 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0c005020 con 0x7eff1c102570 2024-09-19T04:08:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff24758640 1 -- 172.21.15.19:0/2146217388 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c1a6ca0 con 0x7eff1c102570 2024-09-19T04:08:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff24758640 1 -- 172.21.15.19:0/2146217388 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c1a7210 con 0x7eff1c102570 2024-09-19T04:08:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.667+0000 7eff137fe640 1 -- 172.21.15.19:0/2146217388 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0049e0 con 0x7eff1c102570 2024-09-19T04:08:42.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.669+0000 7eff137fe640 1 -- 172.21.15.19:0/2146217388 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7eff0c005560 con 0x7eff1c102570 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff26b31740 1 Processor -- start 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff26b31740 1 -- start start 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff26b31740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02c850 0x56170d02cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff26b31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d0374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff26b31740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d037a30 0x56170d039e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff26b31740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56170cf91160 con 0x56170d02c850 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff26b31740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56170cf38720 con 0x56170d037a30 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff26b31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56170cf8d120 con 0x56170d02d170 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.670+0000 7eff137fe640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7eff080a32c0 0x7eff080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff224f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d0374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff22cf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02c850 0x56170d02cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff22cf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02c850 0x56170d02cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55020/0 (socket says 172.21.15.19:55020) 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff22cf5640 1 -- 172.21.15.19:0/49276596 learned_addr learned my addr 172.21.15.19:0/49276596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff22cf5640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7eff080a32c0 0x7eff080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff137fe640 1 -- 172.21.15.19:0/2146217388 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7eff0c0cfc10 con 0x7eff1c102570 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff21cf3640 1 --2- 172.21.15.19:0/49276596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d037a30 0x56170d039e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff224f4640 1 -- 172.21.15.19:0/49276596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d037a30 msgr2=0x56170d039e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff224f4640 1 --2- 172.21.15.19:0/49276596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d037a30 0x56170d039e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff224f4640 1 -- 172.21.15.19:0/49276596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02c850 msgr2=0x56170d02cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff224f4640 1 --2- 172.21.15.19:0/49276596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02c850 0x56170d02cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff224f4640 1 -- 172.21.15.19:0/49276596 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170cd02ba0 con 0x56170d02d170 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff22cf5640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7eff080a32c0 0x7eff080a5780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7eff1402f480 tx=0x7eff140095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff224f4640 1 --2- 172.21.15.19:0/49276596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d0374f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7eff18009a50 tx=0x7eff18009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47559e05ee9e0f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.671+0000 7eff10ff9640 1 -- 172.21.15.19:0/49276596 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18017070 con 0x56170d02d170 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff10ff9640 1 -- 172.21.15.19:0/49276596 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff18013470 con 0x56170d02d170 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff10ff9640 1 -- 172.21.15.19:0/49276596 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18012610 con 0x56170d02d170 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 -- 172.21.15.19:0/49276596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 msgr2=0x56170d0374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 --2- 172.21.15.19:0/49276596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d0374f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7eff18009a50 tx=0x7eff18009f10 comp rx=0 tx=0).stop 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 -- 172.21.15.19:0/49276596 shutdown_connections 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 --2- 172.21.15.19:0/49276596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d037a30 0x56170d039e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 --2- 172.21.15.19:0/49276596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d0374f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 --2- 172.21.15.19:0/49276596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02c850 0x56170d02cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 -- 172.21.15.19:0/49276596 >> 172.21.15.19:0/49276596 conn(0x56170d028de0 msgr2=0x56170d0291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 -- 172.21.15.19:0/49276596 shutdown_connections 2024-09-19T04:08:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.672+0000 7eff26b31740 1 -- 172.21.15.19:0/49276596 wait complete. 2024-09-19T04:08:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 1 -- start start 2024-09-19T04:08:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 1 Processor -- start 2024-09-19T04:08:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 0x56170d0348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02d170 0x56170d034de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d037a30 0x56170d035320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56170d03a600 con 0x56170d02d170 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56170cf19900 con 0x56170d02c850 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56170d03a780 con 0x56170d037a30 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff26b31740 10 client.0 _ll_register_callbacks cb 0x56170cf17780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff21cf3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02d170 0x56170d034de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff21cf3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02d170 0x56170d034de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55028/0 (socket says 172.21.15.19:55028) 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff21cf3640 1 -- 172.21.15.19:0/3572602238 learned_addr learned my addr 172.21.15.19:0/3572602238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff22cf5640 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d037a30 0x56170d035320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.673+0000 7eff224f4640 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 0x56170d0348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff21cf3640 1 -- 172.21.15.19:0/3572602238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d037a30 msgr2=0x56170d035320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff21cf3640 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d037a30 0x56170d035320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff21cf3640 1 -- 172.21.15.19:0/3572602238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 msgr2=0x56170d0348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff21cf3640 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 0x56170d0348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff21cf3640 1 -- 172.21.15.19:0/3572602238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56170cf59c50 con 0x56170d02d170 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff21cf3640 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02d170 0x56170d034de0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7eff0c004770 tx=0x7eff0c09b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6b14204c41d951 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff05ffb640 1 -- 172.21.15.19:0/3572602238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c01c040 con 0x56170d02d170 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff05ffb640 1 -- 172.21.15.19:0/3572602238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7eff0c004c20 con 0x56170d02d170 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff26b31740 1 -- 172.21.15.19:0/3572602238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff18013470 con 0x56170d02d170 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff05ffb640 1 -- 172.21.15.19:0/3572602238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff0c004e40 con 0x56170d02d170 2024-09-19T04:08:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff26b31740 10 client.15657 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.674+0000 7eff26b31740 1 -- 172.21.15.19:0/3572602238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56170d046290 con 0x56170d02d170 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff05ffb640 1 -- 172.21.15.19:0/3572602238 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff18013470 con 0x56170d02d170 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff05ffb640 1 -- 172.21.15.19:0/3572602238 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7eff0c09b2e0 con 0x56170d02d170 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff26b31740 10 client.15657 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff26b31740 10 client.15657 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff26b31740 20 client.15657 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff26b31740 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56170d0476a0 0x56170d049b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff26b31740 4 client.15657 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff26b31740 1 -- 172.21.15.19:0/3572602238 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56170cf9e890 con 0x56170d0476a0 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.675+0000 7eff224f4640 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56170d0476a0 0x56170d049b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.676+0000 7eff05ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.676+0000 7eff224f4640 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56170d0476a0 0x56170d049b10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c744d2bc51be7b3b server_cookie=338762f7acc7fa32 in_seq=0 out_seq=0 2024-09-19T04:08:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.676+0000 7eff05ffb640 1 -- 172.21.15.19:0/3572602238 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56170cf9e890 con 0x56170d0476a0 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.676+0000 7eff05ffb640 10 client.15657 handle_command_reply: tid=0 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.676+0000 7eff26b31740 1 client.15657 shutdown 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 -- 172.21.15.19:0/3572602238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02d170 msgr2=0x56170d034de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02d170 0x56170d034de0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7eff0c004770 tx=0x7eff0c09b7f0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 -- 172.21.15.19:0/3572602238 shutdown_connections 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56170d0476a0 0x56170d049b10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d037a30 0x56170d035320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d02d170 0x56170d034de0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff05ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 --2- 172.21.15.19:0/3572602238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 0x56170d0348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 -- 172.21.15.19:0/3572602238 >> 172.21.15.19:0/3572602238 conn(0x56170d028de0 msgr2=0x56170d02eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 -- 172.21.15.19:0/3572602238 shutdown_connections 2024-09-19T04:08:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 1 -- 172.21.15.19:0/3572602238 wait complete. 2024-09-19T04:08:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.677+0000 7eff26b31740 20 client.15657 trim_cache size 0 max 16384 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff26b31740 1 Processor -- start 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff26b31740 1 -- start start 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff26b31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02c850 0x56170d04f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff26b31740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02d170 0x56170d04f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff26b31740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d031b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff26b31740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1b0ef0 con 0x56170d037a30 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff26b31740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1b2a30 con 0x56170d02d170 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff26b31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1b15c0 con 0x56170d02c850 2024-09-19T04:08:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.694+0000 7eff22cf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d031b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff224f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02c850 0x56170d04f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff22cf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d031b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55036/0 (socket says 172.21.15.19:55036) 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff22cf5640 1 -- 172.21.15.19:0/110429871 learned_addr learned my addr 172.21.15.19:0/110429871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff21cf3640 1 --2- 172.21.15.19:0/110429871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02d170 0x56170d04f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff22cf5640 1 -- 172.21.15.19:0/110429871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02c850 msgr2=0x56170d04f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff22cf5640 1 --2- 172.21.15.19:0/110429871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02c850 0x56170d04f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff22cf5640 1 -- 172.21.15.19:0/110429871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02d170 msgr2=0x56170d04f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff22cf5640 1 --2- 172.21.15.19:0/110429871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02d170 0x56170d04f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff22cf5640 1 -- 172.21.15.19:0/110429871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170d029a70 con 0x56170d037a30 2024-09-19T04:08:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff22cf5640 1 --2- 172.21.15.19:0/110429871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d031b00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7eff1402ff80 tx=0x7eff14041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4f06cbbf55ddda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff07fff640 1 -- 172.21.15.19:0/110429871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14034040 con 0x56170d037a30 2024-09-19T04:08:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff07fff640 1 -- 172.21.15.19:0/110429871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff14003e50 con 0x56170d037a30 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.695+0000 7eff07fff640 1 -- 172.21.15.19:0/110429871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff140566e0 con 0x56170d037a30 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 -- 172.21.15.19:0/110429871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 msgr2=0x56170d031b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 --2- 172.21.15.19:0/110429871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d031b00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7eff1402ff80 tx=0x7eff14041000 comp rx=0 tx=0).stop 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 -- 172.21.15.19:0/110429871 shutdown_connections 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 --2- 172.21.15.19:0/110429871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d031b00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 --2- 172.21.15.19:0/110429871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02d170 0x56170d04f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 --2- 172.21.15.19:0/110429871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02c850 0x56170d04f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 -- 172.21.15.19:0/110429871 >> 172.21.15.19:0/110429871 conn(0x56170d028de0 msgr2=0x56170d036920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 -- 172.21.15.19:0/110429871 shutdown_connections 2024-09-19T04:08:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 -- 172.21.15.19:0/110429871 wait complete. 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.696+0000 7eff26b31740 1 -- start start 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff26b31740 1 Processor -- start 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff26b31740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 0x56170d03bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff26b31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d03c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff26b31740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff26b31740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1b2360 con 0x56170d037a30 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff26b31740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1b1c90 con 0x56170d02c850 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff26b31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1aeed0 con 0x56170d02d170 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff26b31740 10 client.0 _ll_register_callbacks cb 0x7eff0c097ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff22cf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff22cf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d0b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55050/0 (socket says 172.21.15.19:55050) 2024-09-19T04:08:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff22cf5640 1 -- 172.21.15.19:0/3175879955 learned_addr learned my addr 172.21.15.19:0/3175879955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff21cf3640 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d03c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff224f4640 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 0x56170d03bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff22cf5640 1 -- 172.21.15.19:0/3175879955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 msgr2=0x56170d03c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff22cf5640 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d03c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff22cf5640 1 -- 172.21.15.19:0/3175879955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 msgr2=0x56170d03bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff22cf5640 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 0x56170d03bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff22cf5640 1 -- 172.21.15.19:0/3175879955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56170d02f6c0 con 0x56170d037a30 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.697+0000 7eff21cf3640 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d03c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff22cf5640 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d0b13b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7eff14009a60 tx=0x7eff14004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d070cc71f943802 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff05ffb640 1 -- 172.21.15.19:0/3175879955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14034050 con 0x56170d037a30 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff05ffb640 1 -- 172.21.15.19:0/3175879955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7eff14056690 con 0x56170d037a30 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff26b31740 1 -- 172.21.15.19:0/3175879955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff14003e50 con 0x56170d037a30 2024-09-19T04:08:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff05ffb640 1 -- 172.21.15.19:0/3175879955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff14003ca0 con 0x56170d037a30 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff26b31740 10 client.15663 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff26b31740 1 -- 172.21.15.19:0/3175879955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56170d0b2900 con 0x56170d037a30 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff05ffb640 1 -- 172.21.15.19:0/3175879955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff14003e50 con 0x56170d037a30 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff05ffb640 1 -- 172.21.15.19:0/3175879955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7eff14005020 con 0x56170d037a30 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff26b31740 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff26b31740 10 client.15663 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.698+0000 7eff26b31740 20 client.15663 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.699+0000 7eff26b31740 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56170d0b3030 0x56170d0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.699+0000 7eff26b31740 4 client.15663 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.699+0000 7eff26b31740 1 -- 172.21.15.19:0/3175879955 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x56170cf98be0 con 0x56170d0b3030 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.699+0000 7eff224f4640 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56170d0b3030 0x56170d0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.699+0000 7eff224f4640 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56170d0b3030 0x56170d0b5450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34446b581eb5c859 server_cookie=c18455f2a83f4696 in_seq=0 out_seq=0 2024-09-19T04:08:42.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.699+0000 7eff05ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.711+0000 7eff05ffb640 1 -- 172.21.15.19:0/3175879955 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56170cf98be0 con 0x56170d0b3030 2024-09-19T04:08:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.711+0000 7eff05ffb640 10 client.15663 handle_command_reply: tid=0 2024-09-19T04:08:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 client.15663 shutdown 2024-09-19T04:08:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 -- 172.21.15.19:0/3175879955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 msgr2=0x56170d0b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d0b13b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7eff14009a60 tx=0x7eff14004760 comp rx=0 tx=0).stop 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 -- 172.21.15.19:0/3175879955 shutdown_connections 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56170d037a30 0x56170d0b13b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x56170d0b3030 0x56170d0b5450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170d02d170 0x56170d03c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 --2- 172.21.15.19:0/3175879955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56170d02c850 0x56170d03bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 -- 172.21.15.19:0/3175879955 >> 172.21.15.19:0/3175879955 conn(0x56170d028de0 msgr2=0x56170d036920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff05ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 -- 172.21.15.19:0/3175879955 shutdown_connections 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 1 -- 172.21.15.19:0/3175879955 wait complete. 2024-09-19T04:08:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff26b31740 20 client.15663 trim_cache size 0 max 16384 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 29, 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24476, 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 10, 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "6827a314": { 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 10, 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-19T04:08:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.3, 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:42.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:42.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:08:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:08:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff24758640 1 -- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7eff080a32c0 msgr2=0x7eff080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff24758640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7eff080a32c0 0x7eff080a5780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7eff1402f480 tx=0x7eff140095c0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff24758640 1 -- 172.21.15.19:0/2146217388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff1c102570 msgr2=0x7eff1c197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.712+0000 7eff24758640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff1c102570 0x7eff1c197680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00e9f0 tx=0x7eff0c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.713+0000 7eff24758640 1 -- 172.21.15.19:0/2146217388 shutdown_connections 2024-09-19T04:08:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.713+0000 7eff24758640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7eff080a32c0 0x7eff080a5780 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.713+0000 7eff24758640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff1c102ef0 0x7eff1c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.713+0000 7eff24758640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff1c102570 0x7eff1c197680 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.713+0000 7eff24758640 1 --2- 172.21.15.19:0/2146217388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff1c100e20 0x7eff1c197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.713+0000 7eff24758640 1 -- 172.21.15.19:0/2146217388 >> 172.21.15.19:0/2146217388 conn(0x7eff1c0fc7d0 msgr2=0x7eff1c106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.713+0000 7eff24758640 1 -- 172.21.15.19:0/2146217388 shutdown_connections 2024-09-19T04:08:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:08:42.713+0000 7eff24758640 1 -- 172.21.15.19:0/2146217388 wait complete. 2024-09-19T04:08:43.407 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6827a314', quiesce took 3.2 seconds. Will release after: 17.150000000000002 2024-09-19T04:08:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: pgmap v414: 129 pgs: 129 active+clean; 1.1 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 35 KiB/s wr, 360 op/s 2024-09-19T04:08:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:44.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:44.406 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: pgmap v414: 129 pgs: 129 active+clean; 1.1 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 35 KiB/s wr, 360 op/s 2024-09-19T04:08:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:44.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:44.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: pgmap v414: 129 pgs: 129 active+clean; 1.1 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 35 KiB/s wr, 360 op/s 2024-09-19T04:08:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:08:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:08:44.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:08:44.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:08:46.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:46 smithi111 ceph-mon[35365]: pgmap v415: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 36 KiB/s wr, 362 op/s 2024-09-19T04:08:46.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:46 smithi019 ceph-mon[30938]: pgmap v415: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 36 KiB/s wr, 362 op/s 2024-09-19T04:08:46.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:46 smithi196 ceph-mon[35731]: pgmap v415: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 36 KiB/s wr, 362 op/s 2024-09-19T04:08:48.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:48 smithi111 ceph-mon[35365]: pgmap v416: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 36 KiB/s wr, 361 op/s 2024-09-19T04:08:48.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:48 smithi019 ceph-mon[30938]: pgmap v416: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 36 KiB/s wr, 361 op/s 2024-09-19T04:08:48.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:48 smithi196 ceph-mon[35731]: pgmap v416: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 36 KiB/s wr, 361 op/s 2024-09-19T04:08:49.001 INFO:tasks.workunit.client.0.smithi019.stdout:.......... 2024-09-19T04:08:49.001 INFO:tasks.workunit.client.0.smithi019.stdout:do_o_direct_reads: SUCCESS 2024-09-19T04:08:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 -- 172.21.15.19:0/1117118327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 msgr2=0x7f6c94107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 --2- 172.21.15.19:0/1117118327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 0x7f6c94107300 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009900 tx=0x7f6c8802a410 comp rx=0 tx=0).stop 2024-09-19T04:08:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 -- 172.21.15.19:0/1117118327 shutdown_connections 2024-09-19T04:08:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 --2- 172.21.15.19:0/1117118327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c94107840 0x7f6c94109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 --2- 172.21.15.19:0/1117118327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 0x7f6c94107300 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 --2- 172.21.15.19:0/1117118327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c941025e0 0x7f6c941049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 -- 172.21.15.19:0/1117118327 >> 172.21.15.19:0/1117118327 conn(0x7f6c940fc1e0 msgr2=0x7f6c940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 -- 172.21.15.19:0/1117118327 shutdown_connections 2024-09-19T04:08:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.608+0000 7f6c991bd640 1 -- 172.21.15.19:0/1117118327 wait complete. 2024-09-19T04:08:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.609+0000 7f6c991bd640 1 Processor -- start 2024-09-19T04:08:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.609+0000 7f6c991bd640 1 -- start start 2024-09-19T04:08:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c991bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c941025e0 0x7f6c94197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c991bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 0x7f6c94198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c991bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c94107840 0x7f6c9419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c991bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9410d7f0 con 0x7f6c94104f10 2024-09-19T04:08:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c991bd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9410d670 con 0x7f6c94107840 2024-09-19T04:08:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c991bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9410d970 con 0x7f6c941025e0 2024-09-19T04:08:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 0x7f6c94198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 0x7f6c94198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55072/0 (socket says 172.21.15.19:55072) 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 -- 172.21.15.19:0/2391382465 learned_addr learned my addr 172.21.15.19:0/2391382465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9359e640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c94107840 0x7f6c9419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c941025e0 0x7f6c94197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 -- 172.21.15.19:0/2391382465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c941025e0 msgr2=0x7f6c94197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c941025e0 0x7f6c94197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 -- 172.21.15.19:0/2391382465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c94107840 msgr2=0x7f6c9419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c94107840 0x7f6c9419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 -- 172.21.15.19:0/2391382465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9419fa70 con 0x7f6c94104f10 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c9259c640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 0x7f6c94198390 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6c88031610 tx=0x7f6c88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c7bfff640 1 -- 172.21.15.19:0/2391382465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8803f030 con 0x7f6c94104f10 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c991bd640 1 -- 172.21.15.19:0/2391382465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9419fd00 con 0x7f6c94104f10 2024-09-19T04:08:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.610+0000 7f6c7bfff640 1 -- 172.21.15.19:0/2391382465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c88003ad0 con 0x7f6c94104f10 2024-09-19T04:08:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.611+0000 7f6c7bfff640 1 -- 172.21.15.19:0/2391382465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88035070 con 0x7f6c94104f10 2024-09-19T04:08:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.611+0000 7f6c991bd640 1 -- 172.21.15.19:0/2391382465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c941a01a0 con 0x7f6c94104f10 2024-09-19T04:08:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.613+0000 7f6c7bfff640 1 -- 172.21.15.19:0/2391382465 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6c88042030 con 0x7f6c94104f10 2024-09-19T04:08:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9c034740 1 Processor -- start 2024-09-19T04:08:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9c034740 1 -- start start 2024-09-19T04:08:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b8c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b93c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9c034740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6aeb2f0 con 0x55dfc6b91780 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9c034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6a93270 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9c034740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6ae72b0 con 0x55dfc6b8c6f0 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b93c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b93c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55088/0 (socket says 172.21.15.19:55088) 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9259c640 1 -- 172.21.15.19:0/2071774902 learned_addr learned my addr 172.21.15.19:0/2071774902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9359e640 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b8c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c7bfff640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6c7c0a32f0 0x7f6c7c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9259c640 1 -- 172.21.15.19:0/2071774902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 msgr2=0x55dfc6b911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9259c640 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b911b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9259c640 1 -- 172.21.15.19:0/2071774902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 msgr2=0x55dfc6b8c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9259c640 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b8c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9259c640 1 -- 172.21.15.19:0/2071774902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc6a92340 con 0x55dfc6b91780 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.614+0000 7f6c9359e640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6c7c0a32f0 0x7f6c7c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.615+0000 7f6c7bfff640 1 -- 172.21.15.19:0/2391382465 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6c880f0100 con 0x7f6c94104f10 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.615+0000 7f6c9259c640 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b93c40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6c880e7db0 tx=0x7f6c880e7150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d1ff699a6a8dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.615+0000 7f6c797fa640 1 -- 172.21.15.19:0/2071774902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88049070 con 0x55dfc6b91780 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.615+0000 7f6c797fa640 1 -- 172.21.15.19:0/2071774902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8808f7b0 con 0x55dfc6b91780 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.615+0000 7f6c9359e640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6c7c0a32f0 0x7f6c7c0a57b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6c9406b3e0 tx=0x7f6c8c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.615+0000 7f6c797fa640 1 -- 172.21.15.19:0/2071774902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c880bbd90 con 0x55dfc6b91780 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- 172.21.15.19:0/2071774902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 msgr2=0x55dfc6b93c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b93c40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6c880e7db0 tx=0x7f6c880e7150 comp rx=0 tx=0).stop 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- 172.21.15.19:0/2071774902 shutdown_connections 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b93c40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 --2- 172.21.15.19:0/2071774902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b8c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- 172.21.15.19:0/2071774902 >> 172.21.15.19:0/2071774902 conn(0x55dfc6b83010 msgr2=0x55dfc6b83410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- 172.21.15.19:0/2071774902 shutdown_connections 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- 172.21.15.19:0/2071774902 wait complete. 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- start start 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 Processor -- start 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b95c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b91780 0x55dfc6b9d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6b94510 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6a74290 con 0x55dfc6b91780 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6b94690 con 0x55dfc6b8c6f0 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c9c034740 10 client.0 _ll_register_callbacks cb 0x55dfc6aa5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b95c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b95c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55098/0 (socket says 172.21.15.19:55098) 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.616+0000 7f6c92d9d640 1 -- 172.21.15.19:0/159939052 learned_addr learned my addr 172.21.15.19:0/159939052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c9359e640 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b91780 0x55dfc6b9d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c9259c640 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c92d9d640 1 -- 172.21.15.19:0/159939052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 msgr2=0x55dfc6b961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c92d9d640 1 -- 172.21.15.19:0/159939052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b91780 msgr2=0x55dfc6b9d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b91780 0x55dfc6b9d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c92d9d640 1 -- 172.21.15.19:0/159939052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc67fe210 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c9259c640 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b95c70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6c84012700 tx=0x7f6c84012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264a9eecc0e5a070 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c76ff5640 1 -- 172.21.15.19:0/159939052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84004050 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c76ff5640 1 -- 172.21.15.19:0/159939052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6c840041f0 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c9c034740 1 -- 172.21.15.19:0/159939052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c8808f7b0 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c76ff5640 1 -- 172.21.15.19:0/159939052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c840043f0 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c9c034740 10 client.15675 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c9c034740 1 -- 172.21.15.19:0/159939052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfc68ca620 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.617+0000 7f6c76ff5640 1 -- 172.21.15.19:0/159939052 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8808f7b0 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c76ff5640 1 -- 172.21.15.19:0/159939052 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6c84003c10 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c9c034740 10 client.15675 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c9c034740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c9c034740 20 client.15675 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c9c034740 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55dfc6ba1990 0x55dfc6ba3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c9c034740 4 client.15675 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c9c034740 1 -- 172.21.15.19:0/159939052 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfc6af8a20 con 0x55dfc6ba1990 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c9259c640 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55dfc6ba1990 0x55dfc6ba3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c76ff5640 10 client.15675 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.618+0000 7f6c9259c640 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55dfc6ba1990 0x55dfc6ba3e00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72def1e14a3a0dc2 server_cookie=491499c8aaa75f04 in_seq=0 out_seq=0 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c76ff5640 1 -- 172.21.15.19:0/159939052 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfc6af8a20 con 0x55dfc6ba1990 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c76ff5640 10 client.15675 handle_command_reply: tid=0 2024-09-19T04:08:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 client.15675 shutdown 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 -- 172.21.15.19:0/159939052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b8bdb0 msgr2=0x55dfc6b95c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b95c70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6c84012700 tx=0x7f6c84012bc0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 -- 172.21.15.19:0/159939052 shutdown_connections 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b91780 0x55dfc6b9d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55dfc6ba1990 0x55dfc6ba3e00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 --2- 172.21.15.19:0/159939052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b95c70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c76ff5640 0 client.15675 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 -- 172.21.15.19:0/159939052 >> 172.21.15.19:0/159939052 conn(0x55dfc6b83010 msgr2=0x55dfc6b84820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 -- 172.21.15.19:0/159939052 shutdown_connections 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 1 -- 172.21.15.19:0/159939052 wait complete. 2024-09-19T04:08:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.619+0000 7f6c9c034740 20 client.15675 trim_cache size 0 max 16384 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9c034740 1 Processor -- start 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9c034740 1 -- start start 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b84400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9c034740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941a85a0 con 0x55dfc6b91780 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9c034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941aa0e0 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9c034740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941a8c70 con 0x55dfc6b8c6f0 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b84400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b84400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55102/0 (socket says 172.21.15.19:55102) 2024-09-19T04:08:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 -- 172.21.15.19:0/2448134719 learned_addr learned my addr 172.21.15.19:0/2448134719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9259c640 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 -- 172.21.15.19:0/2448134719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 msgr2=0x55dfc6b9d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b9d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 -- 172.21.15.19:0/2448134719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 msgr2=0x55dfc6b96090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b96090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 -- 172.21.15.19:0/2448134719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc6b957b0 con 0x55dfc6b91780 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c9359e640 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b84400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c012790 tx=0x7f6c8c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db3123cd8192ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.634+0000 7f6c78ff9640 1 -- 172.21.15.19:0/2448134719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c012e50 con 0x55dfc6b91780 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c78ff9640 1 -- 172.21.15.19:0/2448134719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8c010400 con 0x55dfc6b91780 2024-09-19T04:08:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c78ff9640 1 -- 172.21.15.19:0/2448134719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c011640 con 0x55dfc6b91780 2024-09-19T04:08:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 -- 172.21.15.19:0/2448134719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 msgr2=0x55dfc6b84400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b84400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c012790 tx=0x7f6c8c016000 comp rx=0 tx=0).stop 2024-09-19T04:08:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 -- 172.21.15.19:0/2448134719 shutdown_connections 2024-09-19T04:08:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b84400 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b9d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 --2- 172.21.15.19:0/2448134719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b96090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 -- 172.21.15.19:0/2448134719 >> 172.21.15.19:0/2448134719 conn(0x55dfc6b83010 msgr2=0x55dfc6b821c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 -- 172.21.15.19:0/2448134719 shutdown_connections 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 -- 172.21.15.19:0/2448134719 wait complete. 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 -- start start 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.635+0000 7f6c9c034740 1 Processor -- start 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b9ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b9b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9c034740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b9c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9c034740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941a9a10 con 0x55dfc6b91780 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9c034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941a9340 con 0x55dfc6b8c6f0 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9c034740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941a6580 con 0x55dfc6b8bdb0 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9c034740 10 client.0 _ll_register_callbacks cb 0x7f6c840097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b9c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b9c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55114/0 (socket says 172.21.15.19:55114) 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 -- 172.21.15.19:0/1273909209 learned_addr learned my addr 172.21.15.19:0/1273909209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9259c640 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b9b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b9ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 -- 172.21.15.19:0/1273909209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8bdb0 msgr2=0x55dfc6b9ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b9ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 -- 172.21.15.19:0/1273909209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8c6f0 msgr2=0x55dfc6b9b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b9b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 -- 172.21.15.19:0/1273909209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc6b9c940 con 0x55dfc6b91780 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b9ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c9359e640 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b9c330 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c010b70 tx=0x7f6c8c0107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7947e8f0db28bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.636+0000 7f6c76ff5640 1 -- 172.21.15.19:0/1273909209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c021da0 con 0x55dfc6b91780 2024-09-19T04:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c76ff5640 1 -- 172.21.15.19:0/1273909209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6c8c02a3f0 con 0x55dfc6b91780 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 1 -- 172.21.15.19:0/1273909209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c8c010400 con 0x55dfc6b91780 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c76ff5640 1 -- 172.21.15.19:0/1273909209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c8c02a610 con 0x55dfc6b91780 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 10 client.15681 fetch_fsmap learned FSMap version 29 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 1 -- 172.21.15.19:0/1273909209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfc6c0cc40 con 0x55dfc6b91780 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c76ff5640 1 -- 172.21.15.19:0/1273909209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8c010400 con 0x55dfc6b91780 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c76ff5640 1 -- 172.21.15.19:0/1273909209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6c8c00f040 con 0x55dfc6b91780 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 10 client.15681 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 20 client.15681 populate_metadata read hostname 'smithi019' 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55dfc6c0d5e0 0x55dfc6c0fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 4 client.15681 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c9c034740 1 -- 172.21.15.19:0/1273909209 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfc6af2d70 con 0x55dfc6c0d5e0 2024-09-19T04:08:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.637+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55dfc6c0d5e0 0x55dfc6c0fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:08:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.638+0000 7f6c92d9d640 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55dfc6c0d5e0 0x55dfc6c0fa00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bdb2e1096290b4 server_cookie=1c7c9ceb0051fbcd in_seq=0 out_seq=0 2024-09-19T04:08:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.638+0000 7f6c76ff5640 10 client.15681 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c76ff5640 1 -- 172.21.15.19:0/1273909209 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfc6af2d70 con 0x55dfc6c0d5e0 2024-09-19T04:08:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c76ff5640 10 client.15681 handle_command_reply: tid=0 2024-09-19T04:08:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 client.15681 shutdown 2024-09-19T04:08:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 -- 172.21.15.19:0/1273909209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 msgr2=0x55dfc6b9c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b9c330 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c010b70 tx=0x7f6c8c0107e0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 -- 172.21.15.19:0/1273909209 shutdown_connections 2024-09-19T04:08:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dfc6b91780 0x55dfc6b9c330 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dfc6b8c6f0 0x55dfc6b9b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x55dfc6c0d5e0 0x55dfc6c0fa00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 --2- 172.21.15.19:0/1273909209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfc6b8bdb0 0x55dfc6b9ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c76ff5640 0 client.15681 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 -- 172.21.15.19:0/1273909209 >> 172.21.15.19:0/1273909209 conn(0x55dfc6b83010 msgr2=0x55dfc6b833f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.646+0000 7f6c9c034740 1 -- 172.21.15.19:0/1273909209 shutdown_connections 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c9c034740 1 -- 172.21.15.19:0/1273909209 wait complete. 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c9c034740 20 client.15681 trim_cache size 0 max 16384 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:08:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:08:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:08:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 -- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6c7c0a32f0 msgr2=0x7f6c7c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6c7c0a32f0 0x7f6c7c0a57b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6c9406b3e0 tx=0x7f6c8c004a40 comp rx=0 tx=0).stop 2024-09-19T04:08:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 -- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 msgr2=0x7f6c94198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 0x7f6c94198390 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6c88031610 tx=0x7f6c88004270 comp rx=0 tx=0).stop 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 -- 172.21.15.19:0/2391382465 shutdown_connections 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6c7c0a32f0 0x7f6c7c0a57b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c94107840 0x7f6c9419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c94104f10 0x7f6c94198390 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 --2- 172.21.15.19:0/2391382465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c941025e0 0x7f6c94197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 -- 172.21.15.19:0/2391382465 >> 172.21.15.19:0/2391382465 conn(0x7f6c940fc1e0 msgr2=0x7f6c94103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 -- 172.21.15.19:0/2391382465 shutdown_connections 2024-09-19T04:08:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:08:49.647+0000 7f6c991bd640 1 -- 172.21.15.19:0/2391382465 wait complete. 2024-09-19T04:08:50.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:50 smithi111 ceph-mon[35365]: pgmap v417: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 37 KiB/s wr, 1.00k op/s 2024-09-19T04:08:50.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:50 smithi019 ceph-mon[30938]: pgmap v417: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 37 KiB/s wr, 1.00k op/s 2024-09-19T04:08:50.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:50 smithi196 ceph-mon[35731]: pgmap v417: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 37 KiB/s wr, 1.00k op/s 2024-09-19T04:08:52.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:52 smithi111 ceph-mon[35365]: pgmap v418: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 36 KiB/s wr, 999 op/s 2024-09-19T04:08:52.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:52 smithi019 ceph-mon[30938]: pgmap v418: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 36 KiB/s wr, 999 op/s 2024-09-19T04:08:52.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:52 smithi196 ceph-mon[35731]: pgmap v418: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 36 KiB/s wr, 999 op/s 2024-09-19T04:08:54.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:54 smithi111 ceph-mon[35365]: pgmap v419: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 36 KiB/s wr, 1.23k op/s 2024-09-19T04:08:54.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:54 smithi019 ceph-mon[30938]: pgmap v419: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 36 KiB/s wr, 1.23k op/s 2024-09-19T04:08:54.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:54 smithi196 ceph-mon[35731]: pgmap v419: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 36 KiB/s wr, 1.23k op/s 2024-09-19T04:08:56.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:56 smithi111 ceph-mon[35365]: pgmap v420: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 KiB/s wr, 887 op/s 2024-09-19T04:08:56.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:56 smithi019 ceph-mon[30938]: pgmap v420: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 KiB/s wr, 887 op/s 2024-09-19T04:08:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:56 smithi196 ceph-mon[35731]: pgmap v420: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 KiB/s wr, 887 op/s 2024-09-19T04:08:58.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:08:58 smithi111 ceph-mon[35365]: pgmap v421: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 KiB/s wr, 885 op/s 2024-09-19T04:08:58.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:08:58 smithi019 ceph-mon[30938]: pgmap v421: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 KiB/s wr, 885 op/s 2024-09-19T04:08:58.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:08:58 smithi196 ceph-mon[35731]: pgmap v421: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 KiB/s wr, 885 op/s 2024-09-19T04:09:00.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:00 smithi111 ceph-mon[35365]: pgmap v422: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 KiB/s wr, 886 op/s 2024-09-19T04:09:00.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:00 smithi019 ceph-mon[30938]: pgmap v422: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 KiB/s wr, 886 op/s 2024-09-19T04:09:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:00 smithi196 ceph-mon[35731]: pgmap v422: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 KiB/s wr, 886 op/s 2024-09-19T04:09:00.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:09:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:01.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 -- 172.21.15.19:0/248564033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4102790 msgr2=0x7fd5c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:01.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 --2- 172.21.15.19:0/248564033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4102790 0x7fd5c41067e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009900 tx=0x7fd5b802a410 comp rx=0 tx=0).stop 2024-09-19T04:09:01.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 -- 172.21.15.19:0/248564033 shutdown_connections 2024-09-19T04:09:01.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 --2- 172.21.15.19:0/248564033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5c4106f10 0x7fd5c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 --2- 172.21.15.19:0/248564033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4102790 0x7fd5c41067e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 --2- 172.21.15.19:0/248564033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5c4101de0 0x7fd5c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 -- 172.21.15.19:0/248564033 >> 172.21.15.19:0/248564033 conn(0x7fd5c40fd8c0 msgr2=0x7fd5c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 -- 172.21.15.19:0/248564033 shutdown_connections 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.366+0000 7fd5ca935640 1 -- 172.21.15.19:0/248564033 wait complete. 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5ca935640 1 Processor -- start 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5ca935640 1 -- start start 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5ca935640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5c4101de0 0x7fd5c419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5ca935640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5c4102790 0x7fd5c419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5ca935640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4106f10 0x7fd5c419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5ca935640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c4111fb0 con 0x7fd5c4106f10 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5ca935640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c4111e30 con 0x7fd5c4102790 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5ca935640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c4112130 con 0x7fd5c4101de0 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5c8ed2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4106f10 0x7fd5c419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.367+0000 7fd5c8ed2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4106f10 0x7fd5c419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37674/0 (socket says 172.21.15.19:37674) 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c8ed2640 1 -- 172.21.15.19:0/3666677887 learned_addr learned my addr 172.21.15.19:0/3666677887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c3fff640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5c4101de0 0x7fd5c419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c37fe640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5c4102790 0x7fd5c419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c8ed2640 1 -- 172.21.15.19:0/3666677887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5c4101de0 msgr2=0x7fd5c419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c8ed2640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5c4101de0 0x7fd5c419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c8ed2640 1 -- 172.21.15.19:0/3666677887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5c4102790 msgr2=0x7fd5c419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c8ed2640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5c4102790 0x7fd5c419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c8ed2640 1 -- 172.21.15.19:0/3666677887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c419e6c0 con 0x7fd5c4106f10 2024-09-19T04:09:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c3fff640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5c4101de0 0x7fd5c419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c37fe640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5c4102790 0x7fd5c419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c8ed2640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4106f10 0x7fd5c419df90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00eaa0 tx=0x7fd5bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c17fa640 1 -- 172.21.15.19:0/3666677887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00cdb0 con 0x7fd5c4106f10 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c17fa640 1 -- 172.21.15.19:0/3666677887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc005020 con 0x7fd5c4106f10 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c419e9b0 con 0x7fd5c4106f10 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5c17fa640 1 -- 172.21.15.19:0/3666677887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0116b0 con 0x7fd5c4106f10 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.368+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c41a7320 con 0x7fd5c4106f10 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.369+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c41021c0 con 0x7fd5c4106f10 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.371+0000 7fd5c17fa640 1 -- 172.21.15.19:0/3666677887 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd5bc005560 con 0x7fd5c4106f10 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.371+0000 7fd5c17fa640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5ac0a3240 0x7fd5ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.372+0000 7fd5c3fff640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5ac0a3240 0x7fd5ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.372+0000 7fd5c17fa640 1 -- 172.21.15.19:0/3666677887 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fd5bc0cffd0 con 0x7fd5c4106f10 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.372+0000 7fd5c3fff640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5ac0a3240 0x7fd5ac0a5700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4006fd0 tx=0x7fd5b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.376+0000 7fd5c17fa640 1 -- 172.21.15.19:0/3666677887 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd5bc016030 con 0x7fd5c4106f10 2024-09-19T04:09:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.588+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5c41a7620 con 0x7fd5c4106f10 2024-09-19T04:09:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.589+0000 7fd5c17fa640 1 -- 172.21.15.19:0/3666677887 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7fd5bc09c950 con 0x7fd5c4106f10 2024-09-19T04:09:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:09:01.590 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:09:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:09:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5ac0a3240 msgr2=0x7fd5ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5ac0a3240 0x7fd5ac0a5700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4006fd0 tx=0x7fd5b4008040 comp rx=0 tx=0).stop 2024-09-19T04:09:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4106f10 msgr2=0x7fd5c419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4106f10 0x7fd5c419df90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00eaa0 tx=0x7fd5bc00ef60 comp rx=0 tx=0).stop 2024-09-19T04:09:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 shutdown_connections 2024-09-19T04:09:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd5ac0a3240 0x7fd5ac0a5700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5c4106f10 0x7fd5c419df90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5c4102790 0x7fd5c419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 --2- 172.21.15.19:0/3666677887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5c4101de0 0x7fd5c419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 >> 172.21.15.19:0/3666677887 conn(0x7fd5c40fd8c0 msgr2=0x7fd5c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 shutdown_connections 2024-09-19T04:09:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:01.591+0000 7fd5ca935640 1 -- 172.21.15.19:0/3666677887 wait complete. 2024-09-19T04:09:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:02 smithi111 ceph-mon[35365]: pgmap v423: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 KiB/s wr, 243 op/s 2024-09-19T04:09:02.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:02 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3666677887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:09:02.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:02 smithi019 ceph-mon[30938]: pgmap v423: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 KiB/s wr, 243 op/s 2024-09-19T04:09:02.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:02 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3666677887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:09:02.458 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 6827a314 --release --await' 2024-09-19T04:09:02.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 6827a314 --release --await 2024-09-19T04:09:02.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:02 smithi196 ceph-mon[35731]: pgmap v423: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 KiB/s wr, 243 op/s 2024-09-19T04:09:02.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:02 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3666677887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:09:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.742+0000 7fc790503640 1 -- 172.21.15.19:0/3372469010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7881078e0 msgr2=0x7fc788109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.742+0000 7fc790503640 1 --2- 172.21.15.19:0/3372469010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7881078e0 0x7fc788109cd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc780009ca0 tx=0x7fc78002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.742+0000 7fc790503640 1 -- 172.21.15.19:0/3372469010 shutdown_connections 2024-09-19T04:09:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.742+0000 7fc790503640 1 --2- 172.21.15.19:0/3372469010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7881078e0 0x7fc788109cd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.742+0000 7fc790503640 1 --2- 172.21.15.19:0/3372469010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc788104fb0 0x7fc7881073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.743+0000 7fc790503640 1 --2- 172.21.15.19:0/3372469010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc788102680 0x7fc788104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.743+0000 7fc790503640 1 -- 172.21.15.19:0/3372469010 >> 172.21.15.19:0/3372469010 conn(0x7fc7880fc1e0 msgr2=0x7fc7880fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.743+0000 7fc790503640 1 -- 172.21.15.19:0/3372469010 shutdown_connections 2024-09-19T04:09:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.743+0000 7fc790503640 1 -- 172.21.15.19:0/3372469010 wait complete. 2024-09-19T04:09:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.743+0000 7fc790503640 1 Processor -- start 2024-09-19T04:09:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.743+0000 7fc790503640 1 -- start start 2024-09-19T04:09:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc790503640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc788102680 0x7fc788197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc790503640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc788104fb0 0x7fc788198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc790503640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7881078e0 0x7fc78819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc790503640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d7f0 con 0x7fc788102680 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc790503640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d670 con 0x7fc788104fb0 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc790503640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d970 con 0x7fc7881078e0 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc788102680 0x7fc788197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc788102680 0x7fc788197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37704/0 (socket says 172.21.15.19:37704) 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 -- 172.21.15.19:0/1250170909 learned_addr learned my addr 172.21.15.19:0/1250170909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7881078e0 0x7fc78819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78da9e640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc788104fb0 0x7fc788198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 -- 172.21.15.19:0/1250170909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7881078e0 msgr2=0x7fc78819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7881078e0 0x7fc78819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 -- 172.21.15.19:0/1250170909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc788104fb0 msgr2=0x7fc788198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc788104fb0 0x7fc788198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 -- 172.21.15.19:0/1250170909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78819fa10 con 0x7fc788102680 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc78e29f640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc788102680 0x7fc788197df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc78400e9f0 tx=0x7fc78400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.744+0000 7fc77f7fe640 1 -- 172.21.15.19:0/1250170909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78400cd80 con 0x7fc788102680 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.745+0000 7fc77f7fe640 1 -- 172.21.15.19:0/1250170909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc784005500 con 0x7fc788102680 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.745+0000 7fc790503640 1 -- 172.21.15.19:0/1250170909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78819fd00 con 0x7fc788102680 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.745+0000 7fc790503640 1 -- 172.21.15.19:0/1250170909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7881a01a0 con 0x7fc788102680 2024-09-19T04:09:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.745+0000 7fc77f7fe640 1 -- 172.21.15.19:0/1250170909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784010690 con 0x7fc788102680 2024-09-19T04:09:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.747+0000 7fc77f7fe640 1 -- 172.21.15.19:0/1250170909 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc784010830 con 0x7fc788102680 2024-09-19T04:09:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.747+0000 7fc79329b740 1 Processor -- start 2024-09-19T04:09:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc79329b740 1 -- start start 2024-09-19T04:09:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc79329b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054139a0 0x556805413da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc79329b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054142e0 0x556805418da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc79329b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x55680541b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc79329b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568053772f0 con 0x556805419370 2024-09-19T04:09:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc79329b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55680531f270 con 0x5568054139a0 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc79329b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5568053732b0 con 0x5568054142e0 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78da9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x55680541b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78da9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x55680541b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37716/0 (socket says 172.21.15.19:37716) 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78da9e640 1 -- 172.21.15.19:0/477835391 learned_addr learned my addr 172.21.15.19:0/477835391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054139a0 0x556805413da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78e29f640 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054142e0 0x556805418da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78da9e640 1 -- 172.21.15.19:0/477835391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054142e0 msgr2=0x556805418da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78da9e640 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054142e0 0x556805418da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78da9e640 1 -- 172.21.15.19:0/477835391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054139a0 msgr2=0x556805413da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.748+0000 7fc78da9e640 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054139a0 0x556805413da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.749+0000 7fc78da9e640 1 -- 172.21.15.19:0/477835391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680531e340 con 0x556805419370 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.749+0000 7fc77f7fe640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc7740a3370 0x7fc7740a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.749+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc7740a3370 0x7fc7740a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.749+0000 7fc77f7fe640 1 -- 172.21.15.19:0/1250170909 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fc7840d0060 con 0x7fc788102680 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.749+0000 7fc78da9e640 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x55680541b830 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc778009c60 tx=0x7fc778011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa68f5ec1832eedc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.749+0000 7fc77cbfe640 1 -- 172.21.15.19:0/477835391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778004230 con 0x556805419370 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.749+0000 7fc77cbfe640 1 -- 172.21.15.19:0/477835391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7780043d0 con 0x556805419370 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.749+0000 7fc77cbfe640 1 -- 172.21.15.19:0/477835391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778004db0 con 0x556805419370 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc7740a3370 0x7fc7740a5830 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc780006010 tx=0x7fc780031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 -- 172.21.15.19:0/477835391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 msgr2=0x55680541b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x55680541b830 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc778009c60 tx=0x7fc778011810 comp rx=0 tx=0).stop 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 -- 172.21.15.19:0/477835391 shutdown_connections 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x55680541b830 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054142e0 0x556805418da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 --2- 172.21.15.19:0/477835391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054139a0 0x556805413da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 -- 172.21.15.19:0/477835391 >> 172.21.15.19:0/477835391 conn(0x55680540f010 msgr2=0x55680540f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 -- 172.21.15.19:0/477835391 shutdown_connections 2024-09-19T04:09:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.750+0000 7fc79329b740 1 -- 172.21.15.19:0/477835391 wait complete. 2024-09-19T04:09:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 1 -- start start 2024-09-19T04:09:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 1 Processor -- start 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x55680541db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x55680541e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805425110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55680541c340 con 0x5568054142e0 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556805300290 con 0x556805419370 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55680541c4c0 con 0x5568054139a0 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc79329b740 10 client.0 _ll_register_callbacks cb 0x556805331140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc78da9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x55680541e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.751+0000 7fc78da9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x55680541e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37726/0 (socket says 172.21.15.19:37726) 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78da9e640 1 -- 172.21.15.19:0/2279626993 learned_addr learned my addr 172.21.15.19:0/2279626993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78e29f640 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x55680541db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805425110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78da9e640 1 -- 172.21.15.19:0/2279626993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 msgr2=0x55680541db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78da9e640 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x55680541db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78da9e640 1 -- 172.21.15.19:0/2279626993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 msgr2=0x556805425110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78da9e640 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805425110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78da9e640 1 -- 172.21.15.19:0/2279626993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55680508a210 con 0x5568054142e0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc78da9e640 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x55680541e0d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc778011d10 tx=0x7fc778019b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f097c73538b764 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc771ffb640 1 -- 172.21.15.19:0/2279626993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778028020 con 0x5568054142e0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc79329b740 1 -- 172.21.15.19:0/2279626993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7780043d0 con 0x5568054142e0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc771ffb640 1 -- 172.21.15.19:0/2279626993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fc778024620 con 0x5568054142e0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc771ffb640 1 -- 172.21.15.19:0/2279626993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc778033050 con 0x5568054142e0 2024-09-19T04:09:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc79329b740 10 client.15699 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.752+0000 7fc79329b740 1 -- 172.21.15.19:0/2279626993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556805156620 con 0x5568054142e0 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc771ffb640 1 -- 172.21.15.19:0/2279626993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7780043d0 con 0x5568054142e0 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc771ffb640 1 -- 172.21.15.19:0/2279626993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc778018070 con 0x5568054142e0 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc79329b740 10 client.15699 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc79329b740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc79329b740 20 client.15699 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc79329b740 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5568054296b0 0x55680542baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc79329b740 4 client.15699 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc79329b740 1 -- 172.21.15.19:0/2279626993 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556805384a20 con 0x5568054296b0 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.753+0000 7fc78e29f640 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5568054296b0 0x55680542baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.754+0000 7fc771ffb640 10 client.15699 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.754+0000 7fc78e29f640 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5568054296b0 0x55680542baf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def7360a742073bb server_cookie=b1b4cbf9e2a6a7bd in_seq=0 out_seq=0 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.754+0000 7fc771ffb640 1 -- 172.21.15.19:0/2279626993 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556805384a20 con 0x5568054296b0 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.754+0000 7fc771ffb640 10 client.15699 handle_command_reply: tid=0 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.754+0000 7fc79329b740 1 client.15699 shutdown 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 -- 172.21.15.19:0/2279626993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 msgr2=0x55680541e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x55680541e0d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc778011d10 tx=0x7fc778019b70 comp rx=0 tx=0).stop 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 -- 172.21.15.19:0/2279626993 shutdown_connections 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805425110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x55680541e0d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5568054296b0 0x55680542baf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 --2- 172.21.15.19:0/2279626993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x55680541db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc771ffb640 0 client.15699 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 -- 172.21.15.19:0/2279626993 >> 172.21.15.19:0/2279626993 conn(0x55680540f010 msgr2=0x556805410820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 -- 172.21.15.19:0/2279626993 shutdown_connections 2024-09-19T04:09:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 1 -- 172.21.15.19:0/2279626993 wait complete. 2024-09-19T04:09:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.755+0000 7fc79329b740 20 client.15699 trim_cache size 0 max 16384 2024-09-19T04:09:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.776+0000 7fc79329b740 1 Processor -- start 2024-09-19T04:09:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.776+0000 7fc79329b740 1 -- start start 2024-09-19T04:09:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.776+0000 7fc79329b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805410400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.776+0000 7fc79329b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054142e0 0x5568054250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.776+0000 7fc79329b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x556805410940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.776+0000 7fc79329b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c00fbd0 con 0x556805419370 2024-09-19T04:09:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.776+0000 7fc79329b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c011710 con 0x5568054142e0 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.776+0000 7fc79329b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c0102a0 con 0x5568054139a0 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x556805410940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x556805410940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37736/0 (socket says 172.21.15.19:37736) 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 -- 172.21.15.19:0/2092092601 learned_addr learned my addr 172.21.15.19:0/2092092601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78da9e640 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054142e0 0x5568054250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78e29f640 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805410400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 -- 172.21.15.19:0/2092092601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 msgr2=0x556805410400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805410400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 -- 172.21.15.19:0/2092092601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054142e0 msgr2=0x5568054250d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054142e0 0x5568054250d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 -- 172.21.15.19:0/2092092601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680541c690 con 0x556805419370 2024-09-19T04:09:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78e29f640 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805410400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:09:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x556805410940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc780035000 tx=0x7fc780040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2688c77fa1a0942e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc773fff640 1 -- 172.21.15.19:0/2092092601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780031d90 con 0x556805419370 2024-09-19T04:09:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc773fff640 1 -- 172.21.15.19:0/2092092601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc780036040 con 0x556805419370 2024-09-19T04:09:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.777+0000 7fc773fff640 1 -- 172.21.15.19:0/2092092601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78002fad0 con 0x556805419370 2024-09-19T04:09:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 -- 172.21.15.19:0/2092092601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 msgr2=0x556805410940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x556805410940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc780035000 tx=0x7fc780040000 comp rx=0 tx=0).stop 2024-09-19T04:09:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 -- 172.21.15.19:0/2092092601 shutdown_connections 2024-09-19T04:09:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556805419370 0x556805410940 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5568054142e0 0x5568054250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 --2- 172.21.15.19:0/2092092601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805410400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 -- 172.21.15.19:0/2092092601 >> 172.21.15.19:0/2092092601 conn(0x55680540f010 msgr2=0x55680540e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 -- 172.21.15.19:0/2092092601 shutdown_connections 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 -- 172.21.15.19:0/2092092601 wait complete. 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.778+0000 7fc79329b740 1 -- start start 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc79329b740 1 Processor -- start 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc79329b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805422770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc79329b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x556805422cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc79329b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805423c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc79329b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c011040 con 0x5568054142e0 2024-09-19T04:09:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc79329b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c010970 con 0x556805419370 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc79329b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c00dbb0 con 0x5568054139a0 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc79329b740 10 client.0 _ll_register_callbacks cb 0x7fc778005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x556805422cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x556805422cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37744/0 (socket says 172.21.15.19:37744) 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 -- 172.21.15.19:0/707036339 learned_addr learned my addr 172.21.15.19:0/707036339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78e29f640 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805422770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805423c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 -- 172.21.15.19:0/707036339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 msgr2=0x556805422770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805422770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 -- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 msgr2=0x556805423c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805423c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 -- 172.21.15.19:0/707036339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556805424280 con 0x5568054142e0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78eaa0640 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805423c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc78da9e640 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x556805422cb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55680541f070 tx=0x7fc7780099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab1881feffd19a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.779+0000 7fc771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc771ffb640 1 -- 172.21.15.19:0/707036339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77803c060 con 0x5568054142e0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 1 -- 172.21.15.19:0/707036339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc780036040 con 0x5568054142e0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc771ffb640 1 -- 172.21.15.19:0/707036339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fc778005540 con 0x5568054142e0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc771ffb640 1 -- 172.21.15.19:0/707036339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc778043050 con 0x5568054142e0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 1 -- 172.21.15.19:0/707036339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556805494390 con 0x5568054142e0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc771ffb640 1 -- 172.21.15.19:0/707036339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc780036040 con 0x5568054142e0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc771ffb640 1 -- 172.21.15.19:0/707036339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc778018040 con 0x5568054142e0 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:09:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 20 client.15705 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556805494c50 0x556805497070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 4 client.15705 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:09:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.780+0000 7fc79329b740 1 -- 172.21.15.19:0/707036339 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55680537ed70 con 0x556805494c50 2024-09-19T04:09:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.781+0000 7fc78e29f640 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556805494c50 0x556805497070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.781+0000 7fc78e29f640 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556805494c50 0x556805497070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c5a739afb5027ba server_cookie=3205333f1ebac3d1 in_seq=0 out_seq=0 2024-09-19T04:09:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.781+0000 7fc771ffb640 10 client.15705 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.784+0000 7fc771ffb640 1 -- 172.21.15.19:0/707036339 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55680537ed70 con 0x556805494c50 2024-09-19T04:09:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.784+0000 7fc771ffb640 10 client.15705 handle_command_reply: tid=0 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 client.15705 shutdown 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 -- 172.21.15.19:0/707036339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 msgr2=0x556805422cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x556805422cb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55680541f070 tx=0x7fc7780099b0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 -- 172.21.15.19:0/707036339 shutdown_connections 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556805419370 0x556805423c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568054142e0 0x556805422cb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x556805494c50 0x556805497070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 --2- 172.21.15.19:0/707036339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568054139a0 0x556805422770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc771ffb640 0 client.15705 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 -- 172.21.15.19:0/707036339 >> 172.21.15.19:0/707036339 conn(0x55680540f010 msgr2=0x55680540e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 -- 172.21.15.19:0/707036339 shutdown_connections 2024-09-19T04:09:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 1 -- 172.21.15.19:0/707036339 wait complete. 2024-09-19T04:09:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc79329b740 20 client.15705 trim_cache size 0 max 16384 2024-09-19T04:09:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:09:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:09:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:09:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 -- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc7740a3370 msgr2=0x7fc7740a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc7740a3370 0x7fc7740a5830 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc780006010 tx=0x7fc780031bb0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 -- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc788102680 msgr2=0x7fc788197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc788102680 0x7fc788197df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc78400e9f0 tx=0x7fc78400edf0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 -- 172.21.15.19:0/1250170909 shutdown_connections 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fc7740a3370 0x7fc7740a5830 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7881078e0 0x7fc78819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc788104fb0 0x7fc788198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 --2- 172.21.15.19:0/1250170909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc788102680 0x7fc788197df0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.785+0000 7fc790503640 1 -- 172.21.15.19:0/1250170909 >> 172.21.15.19:0/1250170909 conn(0x7fc7880fc1e0 msgr2=0x7fc788102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.786+0000 7fc790503640 1 -- 172.21.15.19:0/1250170909 shutdown_connections 2024-09-19T04:09:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:02.786+0000 7fc790503640 1 -- 172.21.15.19:0/1250170909 wait complete. 2024-09-19T04:09:03.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.205+0000 7fb068854640 1 -- 172.21.15.19:0/82683075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 msgr2=0x7fb06410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.205+0000 7fb068854640 1 --2- 172.21.15.19:0/82683075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 0x7fb06410e610 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009ca0 tx=0x7fb05c02a950 comp rx=0 tx=0).stop 2024-09-19T04:09:03.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.205+0000 7fb068854640 1 -- 172.21.15.19:0/82683075 shutdown_connections 2024-09-19T04:09:03.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.205+0000 7fb068854640 1 --2- 172.21.15.19:0/82683075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 0x7fb06410e610 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.205+0000 7fb068854640 1 --2- 172.21.15.19:0/82683075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb064103660 0x7fb06410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.205+0000 7fb068854640 1 --2- 172.21.15.19:0/82683075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb064102d40 0x7fb064103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.205+0000 7fb068854640 1 -- 172.21.15.19:0/82683075 >> 172.21.15.19:0/82683075 conn(0x7fb0640fc7a0 msgr2=0x7fb0640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.206+0000 7fb068854640 1 -- 172.21.15.19:0/82683075 shutdown_connections 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.206+0000 7fb068854640 1 -- 172.21.15.19:0/82683075 wait complete. 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.206+0000 7fb068854640 1 Processor -- start 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.206+0000 7fb068854640 1 -- start start 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb068854640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb064102d40 0x7fb0641009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb068854640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb064103660 0x7fb064100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb068854640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 0x7fb064101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb068854640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb064111f30 con 0x7fb06410c170 2024-09-19T04:09:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb068854640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb064111db0 con 0x7fb064103660 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb068854640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641120b0 con 0x7fb064102d40 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb06259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb064102d40 0x7fb0641009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb062d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 0x7fb064101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb06259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb064102d40 0x7fb0641009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48470/0 (socket says 172.21.15.19:48470) 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb06259c640 1 -- 172.21.15.19:0/4181100388 learned_addr learned my addr 172.21.15.19:0/4181100388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb061d9b640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb064103660 0x7fb064100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb062d9d640 1 -- 172.21.15.19:0/4181100388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb064102d40 msgr2=0x7fb0641009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb062d9d640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb064102d40 0x7fb0641009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb062d9d640 1 -- 172.21.15.19:0/4181100388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb064103660 msgr2=0x7fb064100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb062d9d640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb064103660 0x7fb064100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb062d9d640 1 -- 172.21.15.19:0/4181100388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0641a6770 con 0x7fb06410c170 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.207+0000 7fb062d9d640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 0x7fb064101420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb05c005cc0 tx=0x7fb05c02f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:03.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.208+0000 7fb0537fe640 1 -- 172.21.15.19:0/4181100388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c034040 con 0x7fb06410c170 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.208+0000 7fb068854640 1 -- 172.21.15.19:0/4181100388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0641a6a00 con 0x7fb06410c170 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.208+0000 7fb0537fe640 1 -- 172.21.15.19:0/4181100388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c030920 con 0x7fb06410c170 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.208+0000 7fb0537fe640 1 -- 172.21.15.19:0/4181100388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c039430 con 0x7fb06410c170 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.208+0000 7fb068854640 1 -- 172.21.15.19:0/4181100388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0641a6ee0 con 0x7fb06410c170 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.210+0000 7fb0537fe640 1 -- 172.21.15.19:0/4181100388 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb05c037020 con 0x7fb06410c170 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.210+0000 7fb06ac2d740 1 Processor -- start 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.210+0000 7fb06ac2d740 1 -- start start 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.210+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377060 0x559128377440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377980 0x559128381d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x559128384630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5591282dc620 con 0x559128377980 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559128283e50 con 0x559128382240 2024-09-19T04:09:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5591282d85e0 con 0x559128377060 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377980 0x559128381d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377980 0x559128381d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37774/0 (socket says 172.21.15.19:37774) 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 -- 172.21.15.19:0/4088157504 learned_addr learned my addr 172.21.15.19:0/4088157504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb061d9b640 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x559128384630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb061d9b640 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377060 0x559128377440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 -- 172.21.15.19:0/4088157504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377060 msgr2=0x559128377440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377060 0x559128377440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 -- 172.21.15.19:0/4088157504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 msgr2=0x559128384630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x559128384630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 -- 172.21.15.19:0/4088157504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559128274d40 con 0x559128377980 2024-09-19T04:09:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb061d9b640 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x559128384630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb0537fe640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb04c0a3240 0x7fb04c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb06259c640 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377980 0x559128381d00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb05400adb0 tx=0x7fb05400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3b1e9329be18d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb061d9b640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb04c0a3240 0x7fb04c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb0537fe640 1 -- 172.21.15.19:0/4181100388 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fb05c0f0d70 con 0x7fb06410c170 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb050ff9640 1 -- 172.21.15.19:0/4088157504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05401d070 con 0x559128377980 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.211+0000 7fb050ff9640 1 -- 172.21.15.19:0/4088157504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb054010d20 con 0x559128377980 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb050ff9640 1 -- 172.21.15.19:0/4088157504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054018cf0 con 0x559128377980 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb06ac2d740 1 -- 172.21.15.19:0/4088157504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377980 msgr2=0x559128381d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377980 0x559128381d00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb05400adb0 tx=0x7fb05400f790 comp rx=0 tx=0).stop 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb061d9b640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb04c0a3240 0x7fb04c0a5700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x559128374970 tx=0x7fb05800a660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb06ac2d740 1 -- 172.21.15.19:0/4088157504 shutdown_connections 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x559128384630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377980 0x559128381d00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/4088157504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377060 0x559128377440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.212+0000 7fb06ac2d740 1 -- 172.21.15.19:0/4088157504 >> 172.21.15.19:0/4088157504 conn(0x5591283735f0 msgr2=0x5591283739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 -- 172.21.15.19:0/4088157504 shutdown_connections 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 -- 172.21.15.19:0/4088157504 wait complete. 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 -- start start 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 Processor -- start 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912837f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x55912837f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x55912837fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559128384e10 con 0x559128377060 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559128264f10 con 0x559128382240 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559128384f90 con 0x559128377980 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06ac2d740 10 client.0 _ll_register_callbacks cb 0x559128288db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912837f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912837f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37784/0 (socket says 172.21.15.19:37784) 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.213+0000 7fb06259c640 1 -- 172.21.15.19:0/1513386673 learned_addr learned my addr 172.21.15.19:0/1513386673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb061d9b640 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x55912837f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06259c640 1 -- 172.21.15.19:0/1513386673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 msgr2=0x55912837f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06259c640 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x55912837f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06259c640 1 -- 172.21.15.19:0/1513386673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 msgr2=0x55912837fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06259c640 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x55912837fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06259c640 1 -- 172.21.15.19:0/1513386673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559127feb580 con 0x559128377060 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06259c640 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912837f100 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb05401b040 tx=0x7fb054010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f336fa7b717df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb049ffb640 1 -- 172.21.15.19:0/1513386673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05401d040 con 0x559128377060 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb049ffb640 1 -- 172.21.15.19:0/1513386673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fb05400fe40 con 0x559128377060 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06ac2d740 1 -- 172.21.15.19:0/1513386673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb054010d20 con 0x559128377060 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb049ffb640 1 -- 172.21.15.19:0/1513386673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb054031050 con 0x559128377060 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06ac2d740 10 client.15717 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb06ac2d740 1 -- 172.21.15.19:0/1513386673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591280b5960 con 0x559128377060 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.214+0000 7fb049ffb640 1 -- 172.21.15.19:0/1513386673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb054010d20 con 0x559128377060 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb049ffb640 1 -- 172.21.15.19:0/1513386673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb054020030 con 0x559128377060 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb06ac2d740 10 client.15717 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb06ac2d740 10 client.15717 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:09:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb06ac2d740 20 client.15717 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x559128391d20 0x559128394110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb06ac2d740 4 client.15717 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb06ac2d740 1 -- 172.21.15.19:0/1513386673 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591282e9b90 con 0x559128391d20 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb061d9b640 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x559128391d20 0x559128394110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb061d9b640 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x559128391d20 0x559128394110 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ba4d569b6f522c server_cookie=ad631ea5d7588d07 in_seq=0 out_seq=0 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.215+0000 7fb049ffb640 10 client.15717 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb049ffb640 1 -- 172.21.15.19:0/1513386673 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591282e9b90 con 0x559128391d20 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb049ffb640 10 client.15717 handle_command_reply: tid=0 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 client.15717 shutdown 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 -- 172.21.15.19:0/1513386673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 msgr2=0x55912837f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912837f100 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb05401b040 tx=0x7fb054010870 comp rx=0 tx=0).stop 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 -- 172.21.15.19:0/1513386673 shutdown_connections 2024-09-19T04:09:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x55912837fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x559128391d20 0x559128394110 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x55912837f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/1513386673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912837f100 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb049ffb640 0 client.15717 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:03.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 -- 172.21.15.19:0/1513386673 >> 172.21.15.19:0/1513386673 conn(0x5591283735f0 msgr2=0x5591283792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:03.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 -- 172.21.15.19:0/1513386673 shutdown_connections 2024-09-19T04:09:03.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.216+0000 7fb06ac2d740 1 -- 172.21.15.19:0/1513386673 wait complete. 2024-09-19T04:09:03.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.217+0000 7fb06ac2d740 20 client.15717 trim_cache size 0 max 16384 2024-09-19T04:09:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.233+0000 7fb06ac2d740 1 Processor -- start 2024-09-19T04:09:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.233+0000 7fb06ac2d740 1 -- start start 2024-09-19T04:09:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128377060 0x559128372a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x5591283997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128382240 0x559128399d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641b0d10 con 0x559128382240 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641b2850 con 0x559128377060 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641b13e0 con 0x559128377980 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128382240 0x559128399d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128382240 0x559128399d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37792/0 (socket says 172.21.15.19:37792) 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 -- 172.21.15.19:0/2414206410 learned_addr learned my addr 172.21.15.19:0/2414206410 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb061d9b640 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x5591283997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb06259c640 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128377060 0x559128372a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 -- 172.21.15.19:0/2414206410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 msgr2=0x5591283997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x5591283997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 -- 172.21.15.19:0/2414206410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128377060 msgr2=0x559128372a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128377060 0x559128372a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 -- 172.21.15.19:0/2414206410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912837e0b0 con 0x559128382240 2024-09-19T04:09:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb06259c640 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128377060 0x559128372a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.234+0000 7fb062d9d640 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128382240 0x559128399d10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0b9450 tx=0x7fb05c0bcbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12c4bb6418eb71f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb04bfff640 1 -- 172.21.15.19:0/2414206410 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0bcff0 con 0x559128382240 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb04bfff640 1 -- 172.21.15.19:0/2414206410 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c0c13a0 con 0x559128382240 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb04bfff640 1 -- 172.21.15.19:0/2414206410 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c004040 con 0x559128382240 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2414206410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128382240 msgr2=0x559128399d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128382240 0x559128399d10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0b9450 tx=0x7fb05c0bcbf0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2414206410 shutdown_connections 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128382240 0x559128399d10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x5591283997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2414206410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128377060 0x559128372a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2414206410 >> 172.21.15.19:0/2414206410 conn(0x5591283735f0 msgr2=0x559128378ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2414206410 shutdown_connections 2024-09-19T04:09:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.235+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2414206410 wait complete. 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 1 -- start start 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 1 Processor -- start 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912838a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x5591283871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x5591283876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641b2180 con 0x559128377060 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641b1ab0 con 0x559128382240 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641aecf0 con 0x559128377980 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06ac2d740 10 client.0 _ll_register_callbacks cb 0x7fb054019e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912838a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912838a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37802/0 (socket says 172.21.15.19:37802) 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06259c640 1 -- 172.21.15.19:0/2590081376 learned_addr learned my addr 172.21.15.19:0/2590081376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb061d9b640 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x5591283871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb062d9d640 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x5591283876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06259c640 1 -- 172.21.15.19:0/2590081376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 msgr2=0x5591283871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06259c640 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x5591283871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06259c640 1 -- 172.21.15.19:0/2590081376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 msgr2=0x5591283876f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06259c640 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x5591283876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.236+0000 7fb06259c640 1 -- 172.21.15.19:0/2590081376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591283777b0 con 0x559128377060 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb06259c640 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912838a720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb05401b040 tx=0x7fb0540184c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fb5f63107ba30e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb049ffb640 1 -- 172.21.15.19:0/2590081376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054039070 con 0x559128377060 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb049ffb640 1 -- 172.21.15.19:0/2590081376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fb0540108e0 con 0x559128377060 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2590081376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c0c13a0 con 0x559128377060 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb049ffb640 1 -- 172.21.15.19:0/2590081376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb054045050 con 0x559128377060 2024-09-19T04:09:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb06ac2d740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2590081376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591283fd430 con 0x559128377060 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb049ffb640 1 -- 172.21.15.19:0/2590081376 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c0c13a0 con 0x559128377060 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb049ffb640 1 -- 172.21.15.19:0/2590081376 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb054020040 con 0x559128377060 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb06ac2d740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb06ac2d740 10 client.15723 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb06ac2d740 20 client.15723 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.237+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5591283fda90 0x5591283ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.238+0000 7fb06ac2d740 4 client.15723 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6827a314", "release": true, "await": true}] 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.238+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2590081376 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6827a314", "release": true, "await": true}) -- 0x5591282e40a0 con 0x5591283fda90 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.238+0000 7fb061d9b640 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5591283fda90 0x5591283ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.238+0000 7fb061d9b640 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5591283fda90 0x5591283ffeb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a6e942f39cb80ff server_cookie=285af33234e50e in_seq=0 out_seq=0 2024-09-19T04:09:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.238+0000 7fb049ffb640 10 client.15723 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb049ffb640 1 -- 172.21.15.19:0/2590081376 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5591282e40a0 con 0x5591283fda90 2024-09-19T04:09:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb049ffb640 10 client.15723 handle_command_reply: tid=0 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 client.15723 shutdown 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2590081376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 msgr2=0x55912838a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912838a720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb05401b040 tx=0x7fb0540184c0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2590081376 shutdown_connections 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559128382240 0x5591283876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5591283fda90 0x5591283ffeb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559128377980 0x5591283871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 --2- 172.21.15.19:0/2590081376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559128377060 0x55912838a720 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2590081376 >> 172.21.15.19:0/2590081376 conn(0x5591283735f0 msgr2=0x55912838d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb049ffb640 0 client.15723 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.241+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2590081376 shutdown_connections 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb06ac2d740 1 -- 172.21.15.19:0/2590081376 wait complete. 2024-09-19T04:09:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb06ac2d740 20 client.15723 trim_cache size 0 max 16384 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 29, 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24476, 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 12, 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "6827a314": { 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 12, 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:09:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.3, 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:09:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 -- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb04c0a3240 msgr2=0x7fb04c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb04c0a3240 0x7fb04c0a5700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x559128374970 tx=0x7fb05800a660 comp rx=0 tx=0).stop 2024-09-19T04:09:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 -- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 msgr2=0x7fb064101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 0x7fb064101420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb05c005cc0 tx=0x7fb05c02f6a0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 -- 172.21.15.19:0/4181100388 shutdown_connections 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fb04c0a3240 0x7fb04c0a5700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06410c170 0x7fb064101420 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb064103660 0x7fb064100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 --2- 172.21.15.19:0/4181100388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb064102d40 0x7fb0641009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 -- 172.21.15.19:0/4181100388 >> 172.21.15.19:0/4181100388 conn(0x7fb0640fc7a0 msgr2=0x7fb0640fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 -- 172.21.15.19:0/4181100388 shutdown_connections 2024-09-19T04:09:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:03.242+0000 7fb068854640 1 -- 172.21.15.19:0/4181100388 wait complete. 2024-09-19T04:09:03.252 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T04:09:03.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T04:09:03.291 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-09-19T04:09:03.291 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-09-19T04:09:03.953 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6827a314', total seconds elapsed: 23.7 2024-09-19T04:09:04.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:04 smithi019 ceph-mon[30938]: pgmap v424: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1023 B/s wr, 243 op/s 2024-09-19T04:09:04.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:04 smithi111 ceph-mon[35365]: pgmap v424: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1023 B/s wr, 243 op/s 2024-09-19T04:09:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:04 smithi196 ceph-mon[35731]: pgmap v424: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1023 B/s wr, 243 op/s 2024-09-19T04:09:04.781 INFO:tasks.workunit.client.0.smithi019.stdout:test read from hole 2024-09-19T04:09:04.781 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test read from hole' 2024-09-19T04:09:04.781 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-09-19T04:09:04.783 INFO:tasks.workunit.client.0.smithi019.stderr:0+0 records in 2024-09-19T04:09:04.783 INFO:tasks.workunit.client.0.smithi019.stderr:0+0 records out 2024-09-19T04:09:04.783 INFO:tasks.workunit.client.0.smithi019.stderr:0 bytes copied, 4.5807e-05 s, 0.0 kB/s 2024-09-19T04:09:04.783 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-09-19T04:09:04.785 INFO:tasks.workunit.client.0.smithi019.stderr:2+0 records in 2024-09-19T04:09:04.785 INFO:tasks.workunit.client.0.smithi019.stderr:2+0 records out 2024-09-19T04:09:04.785 INFO:tasks.workunit.client.0.smithi019.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00141718 s, 723 kB/s 2024-09-19T04:09:04.785 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-09-19T04:09:04.786 INFO:tasks.workunit.client.0.smithi019.stderr:2+0 records in 2024-09-19T04:09:04.786 INFO:tasks.workunit.client.0.smithi019.stderr:2+0 records out 2024-09-19T04:09:04.786 INFO:tasks.workunit.client.0.smithi019.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 5.4592e-05 s, 18.8 MB/s 2024-09-19T04:09:04.786 INFO:tasks.workunit.client.0.smithi019.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-09-19T04:09:04.787 INFO:tasks.workunit.client.0.smithi019.stdout:other thing 2024-09-19T04:09:04.787 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'other thing' 2024-09-19T04:09:04.788 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-09-19T04:09:04.788 INFO:tasks.workunit.client.0.smithi019.stderr:1+0 records in 2024-09-19T04:09:04.788 INFO:tasks.workunit.client.0.smithi019.stderr:1+0 records out 2024-09-19T04:09:04.788 INFO:tasks.workunit.client.0.smithi019.stderr:500 bytes copied, 5.1414e-05 s, 9.7 MB/s 2024-09-19T04:09:04.788 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-09-19T04:09:04.796 INFO:tasks.workunit.client.0.smithi019.stderr:0+1 records in 2024-09-19T04:09:04.796 INFO:tasks.workunit.client.0.smithi019.stderr:0+1 records out 2024-09-19T04:09:04.796 INFO:tasks.workunit.client.0.smithi019.stderr:500 bytes copied, 0.00525894 s, 95.1 kB/s 2024-09-19T04:09:04.796 INFO:tasks.workunit.client.0.smithi019.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-09-19T04:09:04.802 INFO:tasks.workunit.client.0.smithi019.stderr:0+1 records in 2024-09-19T04:09:04.802 INFO:tasks.workunit.client.0.smithi019.stderr:0+1 records out 2024-09-19T04:09:04.802 INFO:tasks.workunit.client.0.smithi019.stderr:500 bytes copied, 0.00501297 s, 99.7 kB/s 2024-09-19T04:09:04.802 INFO:tasks.workunit.client.0.smithi019.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-09-19T04:09:04.802 INFO:tasks.workunit.client.0.smithi019.stdout:OK 2024-09-19T04:09:04.803 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo OK 2024-09-19T04:09:04.804 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T04:09:04.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T04:09:04.848 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-09-19T04:09:04.848 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-09-19T04:09:04.885 INFO:tasks.workunit.client.0.smithi019.stdout:writing first 3 bytes of 10k file 2024-09-19T04:09:04.886 INFO:tasks.workunit.client.0.smithi019.stdout:reading O_DIRECT 2024-09-19T04:09:04.886 INFO:tasks.workunit.client.0.smithi019.stdout:got 10000 2024-09-19T04:09:04.886 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T04:09:04.886 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T04:09:04.964 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-09-19T04:09:04.964 DEBUG:teuthology.orchestra.run.smithi019:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-09-19T04:09:06.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:06 smithi111 ceph-mon[35365]: pgmap v425: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T04:09:06.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:06 smithi019 ceph-mon[30938]: pgmap v425: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T04:09:06.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:06 smithi196 ceph-mon[35731]: pgmap v425: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T04:09:08.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:08 smithi019 ceph-mon[30938]: pgmap v426: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:09:08.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:08 smithi196 ceph-mon[35731]: pgmap v426: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:09:08.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:08 smithi111 ceph-mon[35365]: pgmap v426: 129 pgs: 129 active+clean; 1.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:09:10.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:10 smithi019 ceph-mon[30938]: pgmap v427: 129 pgs: 129 active+clean; 795 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-09-19T04:09:10.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:10 smithi196 ceph-mon[35731]: pgmap v427: 129 pgs: 129 active+clean; 795 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-09-19T04:09:10.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:10 smithi111 ceph-mon[35365]: pgmap v427: 129 pgs: 129 active+clean; 795 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-09-19T04:09:11.225 INFO:tasks.workunit.client.0.smithi019.stdout:writing pattern 2024-09-19T04:09:11.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-09-19T04:09:11.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-19T04:09:11.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-09-19T04:09:11.225 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-19T04:09:11.226 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-19T04:09:11.227 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-09-19T04:09:11.228 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-19T04:09:11.229 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-19T04:09:11.230 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-19T04:09:11.231 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-09-19T04:09:11.232 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-19T04:09:11.232 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-09-19T04:09:11.232 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-19T04:09:11.232 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-09-19T04:09:11.232 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-19T04:09:11.232 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-09-19T04:09:11.232 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-19T04:09:11.232 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-19T04:09:11.269 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-09-19T04:09:11.270 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-09-19T04:09:11.271 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-19T04:09:11.272 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-19T04:09:11.273 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-09-19T04:09:11.274 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-19T04:09:11.275 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-09-19T04:09:11.275 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-19T04:09:11.275 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-09-19T04:09:11.275 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-19T04:09:11.275 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-09-19T04:09:11.295 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-19T04:09:11.295 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-09-19T04:09:11.295 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-19T04:09:11.295 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-09-19T04:09:11.295 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-19T04:09:11.295 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-19T04:09:11.296 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-19T04:09:11.297 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-19T04:09:11.298 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-19T04:09:11.299 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-09-19T04:09:11.300 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-19T04:09:11.300 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-09-19T04:09:11.300 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-19T04:09:11.300 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-09-19T04:09:11.300 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-19T04:09:11.300 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-09-19T04:09:11.300 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-19T04:09:11.301 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-09-19T04:09:11.301 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-19T04:09:11.301 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-09-19T04:09:11.301 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-19T04:09:11.301 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-09-19T04:09:11.301 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-19T04:09:11.302 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-09-19T04:09:11.303 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-19T04:09:11.303 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-09-19T04:09:11.303 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-19T04:09:11.303 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-09-19T04:09:11.303 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-19T04:09:11.323 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-09-19T04:09:11.324 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-19T04:09:11.325 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-09-19T04:09:11.326 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-19T04:09:11.327 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-09-19T04:09:11.328 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-19T04:09:11.329 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-09-19T04:09:11.330 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-19T04:09:11.330 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-09-19T04:09:11.330 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-19T04:09:11.330 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-09-19T04:09:11.330 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-19T04:09:11.330 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-09-19T04:09:11.330 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-19T04:09:11.330 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-09-19T04:09:11.365 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-19T04:09:11.365 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-09-19T04:09:11.365 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-19T04:09:11.365 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-09-19T04:09:11.365 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-09-19T04:09:11.366 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-09-19T04:09:11.367 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-19T04:09:11.368 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-09-19T04:09:11.369 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-19T04:09:11.370 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-19T04:09:11.371 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-09-19T04:09:11.402 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-19T04:09:11.403 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-09-19T04:09:11.404 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-09-19T04:09:11.405 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-19T04:09:11.406 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-19T04:09:11.407 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-19T04:09:11.408 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-09-19T04:09:11.427 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-19T04:09:11.427 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-09-19T04:09:11.427 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-19T04:09:11.427 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-09-19T04:09:11.427 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-19T04:09:11.427 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-09-19T04:09:11.427 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-19T04:09:11.428 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-19T04:09:11.429 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-09-19T04:09:11.430 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-19T04:09:11.431 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-09-19T04:09:11.432 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-19T04:09:11.433 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-09-19T04:09:11.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-19T04:09:11.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-09-19T04:09:11.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-19T04:09:11.434 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-09-19T04:09:11.456 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-09-19T04:09:11.457 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-09-19T04:09:11.458 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-09-19T04:09:11.459 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-19T04:09:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-09-19T04:09:11.461 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-19T04:09:11.462 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-09-19T04:09:11.462 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-19T04:09:11.462 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-09-19T04:09:11.462 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-19T04:09:11.462 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-09-19T04:09:11.462 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-19T04:09:11.462 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-09-19T04:09:11.462 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-19T04:09:11.463 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-09-19T04:09:11.463 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-19T04:09:11.493 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-09-19T04:09:11.493 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-19T04:09:11.494 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-09-19T04:09:11.494 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-19T04:09:11.494 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-09-19T04:09:11.494 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-19T04:09:11.494 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-09-19T04:09:11.494 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-19T04:09:11.495 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-09-19T04:09:11.496 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-09-19T04:09:11.497 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-19T04:09:11.498 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-09-19T04:09:11.499 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-19T04:09:11.500 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-09-19T04:09:11.501 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-19T04:09:11.501 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-09-19T04:09:11.501 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-19T04:09:11.501 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-09-19T04:09:11.525 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-19T04:09:11.525 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-09-19T04:09:11.525 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-19T04:09:11.525 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-09-19T04:09:11.526 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-19T04:09:11.526 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-09-19T04:09:11.526 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-19T04:09:11.526 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-09-19T04:09:11.526 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-19T04:09:11.526 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-09-19T04:09:11.526 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-09-19T04:09:11.527 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-19T04:09:11.528 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-19T04:09:11.529 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-09-19T04:09:11.530 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-19T04:09:11.530 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-09-19T04:09:11.530 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-19T04:09:11.530 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-09-19T04:09:11.530 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-19T04:09:11.530 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-19T04:09:11.531 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-09-19T04:09:11.532 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-19T04:09:11.533 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-09-19T04:09:11.548 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-19T04:09:11.548 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-09-19T04:09:11.548 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-19T04:09:11.548 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-09-19T04:09:11.548 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-19T04:09:11.548 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-09-19T04:09:11.548 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-09-19T04:09:11.549 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-19T04:09:11.550 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-19T04:09:11.551 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-09-19T04:09:11.552 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-09-19T04:09:11.553 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-09-19T04:09:11.554 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-19T04:09:11.578 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-09-19T04:09:11.578 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-19T04:09:11.578 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-09-19T04:09:11.578 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-19T04:09:11.578 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-19T04:09:11.579 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-09-19T04:09:11.580 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-19T04:09:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-09-19T04:09:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-19T04:09:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-09-19T04:09:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-19T04:09:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-09-19T04:09:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-19T04:09:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-09-19T04:09:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-19T04:09:11.582 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-09-19T04:09:11.582 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-19T04:09:11.582 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-09-19T04:09:11.582 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-19T04:09:11.582 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-09-19T04:09:11.582 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-19T04:09:11.583 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-09-19T04:09:11.583 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-19T04:09:11.583 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-09-19T04:09:11.583 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-19T04:09:11.583 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-09-19T04:09:11.583 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-19T04:09:11.583 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-09-19T04:09:11.584 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-19T04:09:11.584 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-09-19T04:09:11.584 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-19T04:09:11.584 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-09-19T04:09:11.584 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-19T04:09:11.584 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-09-19T04:09:11.584 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-09-19T04:09:11.585 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-19T04:09:11.586 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-09-19T04:09:11.586 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-19T04:09:11.586 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-09-19T04:09:11.586 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-19T04:09:11.586 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-09-19T04:09:11.586 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-19T04:09:11.587 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-09-19T04:09:11.587 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-19T04:09:11.587 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-09-19T04:09:11.587 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-19T04:09:11.587 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-09-19T04:09:11.587 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-19T04:09:11.587 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-09-19T04:09:11.587 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-19T04:09:11.618 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-09-19T04:09:11.618 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-19T04:09:11.618 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-09-19T04:09:11.618 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-19T04:09:11.618 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-09-19T04:09:11.619 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-09-19T04:09:11.620 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-09-19T04:09:11.621 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-09-19T04:09:11.622 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-19T04:09:11.623 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-09-19T04:09:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-19T04:09:11.650 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-09-19T04:09:11.650 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-19T04:09:11.650 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-09-19T04:09:11.650 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-19T04:09:11.650 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-09-19T04:09:11.650 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-19T04:09:11.650 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-09-19T04:09:11.650 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-09-19T04:09:11.651 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-19T04:09:11.652 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-09-19T04:09:11.653 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-19T04:09:11.654 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-09-19T04:09:11.655 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-19T04:09:11.656 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-19T04:09:11.675 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-09-19T04:09:11.676 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-19T04:09:11.677 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-09-19T04:09:11.678 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-09-19T04:09:11.679 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-19T04:09:11.680 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-09-19T04:09:11.681 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-19T04:09:11.681 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-09-19T04:09:11.681 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-19T04:09:11.681 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-09-19T04:09:11.708 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-19T04:09:11.708 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-09-19T04:09:11.708 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-19T04:09:11.708 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-09-19T04:09:11.708 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-19T04:09:11.708 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-09-19T04:09:11.708 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-19T04:09:11.708 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-09-19T04:09:11.709 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-09-19T04:09:11.710 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-09-19T04:09:11.711 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-19T04:09:11.712 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-19T04:09:11.713 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-09-19T04:09:11.714 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-19T04:09:11.750 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-09-19T04:09:11.750 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-19T04:09:11.750 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-09-19T04:09:11.750 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-19T04:09:11.750 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-09-19T04:09:11.750 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-09-19T04:09:11.751 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-19T04:09:11.752 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-09-19T04:09:11.753 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-19T04:09:11.754 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-09-19T04:09:11.755 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-19T04:09:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-09-19T04:09:11.757 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-09-19T04:09:11.780 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-09-19T04:09:11.781 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-19T04:09:11.782 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-09-19T04:09:11.783 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-09-19T04:09:11.784 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-19T04:09:11.785 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-09-19T04:09:11.786 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-19T04:09:11.786 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-09-19T04:09:11.786 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-19T04:09:11.786 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-09-19T04:09:11.786 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-19T04:09:11.806 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-09-19T04:09:11.806 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-19T04:09:11.806 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-09-19T04:09:11.806 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-19T04:09:11.806 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-09-19T04:09:11.806 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-19T04:09:11.806 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-09-19T04:09:11.806 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-19T04:09:11.807 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-19T04:09:11.808 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-09-19T04:09:11.809 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-19T04:09:11.810 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-09-19T04:09:11.810 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-19T04:09:11.810 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-09-19T04:09:11.810 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-19T04:09:11.810 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-09-19T04:09:11.810 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-19T04:09:11.810 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-09-19T04:09:11.810 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-19T04:09:11.811 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-19T04:09:11.812 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-09-19T04:09:11.842 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-09-19T04:09:11.843 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-19T04:09:11.844 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-19T04:09:11.845 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-09-19T04:09:11.846 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-09-19T04:09:11.847 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-19T04:09:11.848 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-09-19T04:09:11.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-19T04:09:11.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-09-19T04:09:11.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-19T04:09:11.885 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-09-19T04:09:11.886 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-19T04:09:11.887 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-09-19T04:09:11.888 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-19T04:09:11.889 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-19T04:09:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-09-19T04:09:11.891 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-19T04:09:11.913 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-09-19T04:09:11.913 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-19T04:09:11.913 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-09-19T04:09:11.913 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-19T04:09:11.914 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-09-19T04:09:11.915 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-19T04:09:11.916 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-19T04:09:11.917 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-09-19T04:09:11.918 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-09-19T04:09:11.919 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-19T04:09:11.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-09-19T04:09:11.920 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-19T04:09:11.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-09-19T04:09:11.939 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-19T04:09:11.940 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-09-19T04:09:11.941 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-19T04:09:11.942 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-09-19T04:09:11.943 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-19T04:09:11.944 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-09-19T04:09:11.945 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-19T04:09:11.946 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-09-19T04:09:11.946 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-19T04:09:11.946 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-09-19T04:09:11.946 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-19T04:09:11.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-09-19T04:09:11.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-19T04:09:11.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-09-19T04:09:11.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-19T04:09:11.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-09-19T04:09:11.979 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-19T04:09:11.980 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-19T04:09:11.981 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-19T04:09:11.982 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-09-19T04:09:11.983 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-09-19T04:09:11.984 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-19T04:09:11.985 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-19T04:09:12.022 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-19T04:09:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-09-19T04:09:12.024 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-19T04:09:12.025 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-19T04:09:12.026 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-09-19T04:09:12.027 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-19T04:09:12.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-09-19T04:09:12.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-19T04:09:12.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-09-19T04:09:12.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-19T04:09:12.028 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-09-19T04:09:12.045 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-19T04:09:12.045 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-09-19T04:09:12.045 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-19T04:09:12.045 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-09-19T04:09:12.045 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-19T04:09:12.045 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-09-19T04:09:12.045 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-19T04:09:12.045 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-19T04:09:12.046 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-09-19T04:09:12.047 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-19T04:09:12.048 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-09-19T04:09:12.049 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-09-19T04:09:12.050 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-19T04:09:12.051 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-09-19T04:09:12.051 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-19T04:09:12.051 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-09-19T04:09:12.051 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-19T04:09:12.051 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-09-19T04:09:12.051 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-19T04:09:12.051 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-09-19T04:09:12.051 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-09-19T04:09:12.068 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-09-19T04:09:12.069 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-19T04:09:12.070 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-19T04:09:12.071 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-19T04:09:12.072 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-19T04:09:12.073 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-09-19T04:09:12.074 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-19T04:09:12.104 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-09-19T04:09:12.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-19T04:09:12.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-09-19T04:09:12.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-19T04:09:12.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-09-19T04:09:12.105 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-19T04:09:12.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-09-19T04:09:12.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-19T04:09:12.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-09-19T04:09:12.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-19T04:09:12.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-09-19T04:09:12.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-19T04:09:12.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-09-19T04:09:12.106 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-19T04:09:12.107 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-09-19T04:09:12.108 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-09-19T04:09:12.109 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-09-19T04:09:12.110 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-09-19T04:09:12.111 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-19T04:09:12.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-09-19T04:09:12.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-19T04:09:12.112 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-09-19T04:09:12.148 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-19T04:09:12.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-09-19T04:09:12.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-19T04:09:12.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-09-19T04:09:12.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-19T04:09:12.149 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-09-19T04:09:12.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-19T04:09:12.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-09-19T04:09:12.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-19T04:09:12.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-09-19T04:09:12.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-19T04:09:12.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-09-19T04:09:12.150 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-19T04:09:12.151 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-09-19T04:09:12.152 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-19T04:09:12.153 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-19T04:09:12.154 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-09-19T04:09:12.155 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-19T04:09:12.156 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-09-19T04:09:12.172 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-09-19T04:09:12.173 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-19T04:09:12.174 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-09-19T04:09:12.175 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-09-19T04:09:12.176 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-09-19T04:09:12.177 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-19T04:09:12.178 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-09-19T04:09:12.179 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-19T04:09:12.179 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-09-19T04:09:12.179 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-19T04:09:12.179 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-09-19T04:09:12.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:12 smithi019 ceph-mon[30938]: pgmap v428: 129 pgs: 129 active+clean; 894 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-19T04:09:12.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:12 smithi196 ceph-mon[35731]: pgmap v428: 129 pgs: 129 active+clean; 894 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-19T04:09:12.478 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-19T04:09:12.478 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-09-19T04:09:12.478 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-19T04:09:12.478 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-09-19T04:09:12.478 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-19T04:09:12.478 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-19T04:09:12.479 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-19T04:09:12.480 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-19T04:09:12.481 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-09-19T04:09:12.482 INFO:tasks.workunit.client.0.smithi019.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-19T04:09:12.483 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-09-19T04:09:12.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-19T04:09:12.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-09-19T04:09:12.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-19T04:09:12.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-09-19T04:09:12.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-19T04:09:12.484 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-09-19T04:09:12.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:12 smithi111 ceph-mon[35365]: pgmap v428: 129 pgs: 129 active+clean; 894 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-09-19T04:09:13.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-09-19T04:09:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-09-19T04:09:13.588 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-19T04:09:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-19T04:09:13.590 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-09-19T04:09:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-19T04:09:13.592 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-09-19T04:09:13.592 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-19T04:09:13.592 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-09-19T04:09:13.592 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-19T04:09:13.592 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-09-19T04:09:13.592 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-19T04:09:13.593 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-09-19T04:09:13.593 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-19T04:09:13.593 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-09-19T04:09:13.593 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-19T04:09:13.593 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-09-19T04:09:13.594 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-19T04:09:13.594 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-09-19T04:09:13.594 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-19T04:09:13.594 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-19T04:09:14.346 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-09-19T04:09:14.347 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-19T04:09:14.347 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-09-19T04:09:14.347 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-19T04:09:14.347 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-09-19T04:09:14.347 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-19T04:09:14.347 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-09-19T04:09:14.347 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-19T04:09:14.348 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-09-19T04:09:14.348 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-19T04:09:14.348 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-09-19T04:09:14.348 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-19T04:09:14.348 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-09-19T04:09:14.348 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-19T04:09:14.348 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-19T04:09:14.349 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-09-19T04:09:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-19T04:09:14.351 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-19T04:09:14.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-09-19T04:09:14.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-19T04:09:14.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-09-19T04:09:14.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-19T04:09:14.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-09-19T04:09:14.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-19T04:09:14.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-09-19T04:09:14.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-19T04:09:14.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:14 smithi019 ceph-mon[30938]: pgmap v429: 129 pgs: 129 active+clean; 890 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-09-19T04:09:14.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:14 smithi196 ceph-mon[35731]: pgmap v429: 129 pgs: 129 active+clean; 890 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-09-19T04:09:14.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:14 smithi111 ceph-mon[35365]: pgmap v429: 129 pgs: 129 active+clean; 890 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-09-19T04:09:14.793 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-09-19T04:09:14.793 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-19T04:09:14.793 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-09-19T04:09:14.793 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-19T04:09:14.793 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-09-19T04:09:14.793 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-19T04:09:14.793 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-09-19T04:09:14.793 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-09-19T04:09:14.794 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-19T04:09:14.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-19T04:09:14.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-09-19T04:09:14.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-19T04:09:14.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-09-19T04:09:14.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-19T04:09:14.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-09-19T04:09:14.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-19T04:09:14.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-19T04:09:14.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-09-19T04:09:14.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-09-19T04:09:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-19T04:09:15.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-09-19T04:09:15.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-19T04:09:15.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-09-19T04:09:15.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-09-19T04:09:15.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-09-19T04:09:15.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-09-19T04:09:15.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-09-19T04:09:15.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-19T04:09:15.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-09-19T04:09:15.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-19T04:09:15.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-09-19T04:09:15.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-19T04:09:15.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-09-19T04:09:15.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-19T04:09:15.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-09-19T04:09:15.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-19T04:09:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 -- 172.21.15.19:0/2309172647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe70101d80 msgr2=0x7fbe70108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 --2- 172.21.15.19:0/2309172647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe70101d80 0x7fbe70108610 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009ca0 tx=0x7fbe6002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 -- 172.21.15.19:0/2309172647 shutdown_connections 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 --2- 172.21.15.19:0/2309172647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe70101d80 0x7fbe70108610 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 --2- 172.21.15.19:0/2309172647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe701013c0 0x7fbe70101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 --2- 172.21.15.19:0/2309172647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe701001d0 0x7fbe701005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 -- 172.21.15.19:0/2309172647 >> 172.21.15.19:0/2309172647 conn(0x7fbe700fb930 msgr2=0x7fbe700fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 -- 172.21.15.19:0/2309172647 shutdown_connections 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 -- 172.21.15.19:0/2309172647 wait complete. 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.886+0000 7fbe7581e640 1 Processor -- start 2024-09-19T04:09:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe7581e640 1 -- start start 2024-09-19T04:09:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe7581e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe701001d0 0x7fbe70199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe7581e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe701013c0 0x7fbe7019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe7581e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe70101d80 0x7fbe701a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe7581e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7010b610 con 0x7fbe701013c0 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe7581e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7010b490 con 0x7fbe701001d0 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe7581e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7010b790 con 0x7fbe70101d80 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe701013c0 0x7fbe7019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe701013c0 0x7fbe7019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45572/0 (socket says 172.21.15.19:45572) 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/825362805 learned_addr learned my addr 172.21.15.19:0/825362805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6effd640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe701001d0 0x7fbe70199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe70101d80 0x7fbe701a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/825362805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe70101d80 msgr2=0x7fbe701a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe70101d80 0x7fbe701a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/825362805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe701001d0 msgr2=0x7fbe70199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe701001d0 0x7fbe70199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/825362805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe701a1b90 con 0x7fbe701013c0 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.887+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe701013c0 0x7fbe7019a4b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800e9f0 tx=0x7fbe5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.888+0000 7fbe7481c640 1 -- 172.21.15.19:0/825362805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800cd80 con 0x7fbe701013c0 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.888+0000 7fbe7481c640 1 -- 172.21.15.19:0/825362805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe58010910 con 0x7fbe701013c0 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.888+0000 7fbe7581e640 1 -- 172.21.15.19:0/825362805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe701a1e80 con 0x7fbe701013c0 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.888+0000 7fbe7481c640 1 -- 172.21.15.19:0/825362805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58018990 con 0x7fbe701013c0 2024-09-19T04:09:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.888+0000 7fbe7581e640 1 -- 172.21.15.19:0/825362805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe701a2340 con 0x7fbe701013c0 2024-09-19T04:09:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.890+0000 7fbe7481c640 1 -- 172.21.15.19:0/825362805 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fbe58010ac0 con 0x7fbe701013c0 2024-09-19T04:09:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7481c640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe5c0a3310 0x7fbe5c0a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7481c640 1 -- 172.21.15.19:0/825362805 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fbe580cffd0 con 0x7fbe701013c0 2024-09-19T04:09:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe6effd640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe5c0a3310 0x7fbe5c0a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7868e740 1 Processor -- start 2024-09-19T04:09:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7868e740 1 -- start start 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d69a0 0x560dbc1d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1dbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1dc370 0x560dbc1de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7868e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560dbc13a2f0 con 0x560dbc1d69a0 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7868e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560dbc0e2270 con 0x560dbc1d72e0 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.891+0000 7fbe7868e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560dbc1362b0 con 0x560dbc1dc370 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1dbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6effd640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe5c0a3310 0x7fbe5c0a57d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe68005fd0 tx=0x7fbe6800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d69a0 0x560dbc1d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1dbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:54156/0 (socket says 172.21.15.19:54156) 2024-09-19T04:09:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/946924781 learned_addr learned my addr 172.21.15.19:0/946924781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/946924781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1dc370 0x560dbc1de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/946924781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1dc370 msgr2=0x560dbc1de830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/946924781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1dc370 0x560dbc1de830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/946924781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d69a0 msgr2=0x560dbc1d6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/946924781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d69a0 0x560dbc1d6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/946924781 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbc0e1340 con 0x560dbc1d72e0 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/946924781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1dbda0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe5809c1e0 tx=0x7fbe5809c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9d25aea5b945d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe65ffb640 1 -- 172.21.15.19:0/946924781 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe580a4ec0 con 0x560dbc1d72e0 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe65ffb640 1 -- 172.21.15.19:0/946924781 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5809a550 con 0x560dbc1d72e0 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.892+0000 7fbe65ffb640 1 -- 172.21.15.19:0/946924781 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe580bc2d0 con 0x560dbc1d72e0 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 -- 172.21.15.19:0/946924781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 msgr2=0x560dbc1dbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 --2- 172.21.15.19:0/946924781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1dbda0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe5809c1e0 tx=0x7fbe5809c950 comp rx=0 tx=0).stop 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 -- 172.21.15.19:0/946924781 shutdown_connections 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 --2- 172.21.15.19:0/946924781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1dc370 0x560dbc1de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 --2- 172.21.15.19:0/946924781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1dbda0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 --2- 172.21.15.19:0/946924781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d69a0 0x560dbc1d6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 -- 172.21.15.19:0/946924781 >> 172.21.15.19:0/946924781 conn(0x560dbc1d2010 msgr2=0x560dbc1d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 -- 172.21.15.19:0/946924781 shutdown_connections 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.893+0000 7fbe7868e740 1 -- 172.21.15.19:0/946924781 wait complete. 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 1 -- start start 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 1 Processor -- start 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d72e0 0x560dbc1e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1dc370 0x560dbc1e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560dbc1df340 con 0x560dbc1d72e0 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560dbc0c3290 con 0x560dbc1dc370 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560dbc1df4c0 con 0x560dbc1d69a0 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe7868e740 10 client.0 _ll_register_callbacks cb 0x560dbc0f4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe6e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d72e0 0x560dbc1e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1dc370 0x560dbc1e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe6e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d72e0 0x560dbc1e11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45592/0 (socket says 172.21.15.19:45592) 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.894+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/2045969998 learned_addr learned my addr 172.21.15.19:0/2045969998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe6effd640 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/2045969998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 msgr2=0x560dbc1e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/2045969998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1dc370 msgr2=0x560dbc1e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1dc370 0x560dbc1e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe6e7fc640 1 -- 172.21.15.19:0/2045969998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbbe4d210 con 0x560dbc1d72e0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d72e0 0x560dbc1e11b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800cfd0 tx=0x7fbe5809a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3403b104ed6537 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe4effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe4effd640 1 -- 172.21.15.19:0/2045969998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe580b5210 con 0x560dbc1d72e0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe4effd640 1 -- 172.21.15.19:0/2045969998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fbe58018d40 con 0x560dbc1d72e0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe7868e740 1 -- 172.21.15.19:0/2045969998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5809a550 con 0x560dbc1d72e0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe4effd640 1 -- 172.21.15.19:0/2045969998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe580a4ec0 con 0x560dbc1d72e0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe7868e740 10 client.15735 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe7868e740 1 -- 172.21.15.19:0/2045969998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560dbbf19620 con 0x560dbc1d72e0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.895+0000 7fbe4effd640 1 -- 172.21.15.19:0/2045969998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5809a550 con 0x560dbc1d72e0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe4effd640 1 -- 172.21.15.19:0/2045969998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbe58098140 con 0x560dbc1d72e0 2024-09-19T04:09:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe7868e740 10 client.15735 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe7868e740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe7868e740 20 client.15735 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560dbc1ec4f0 0x560dbc1ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe7868e740 4 client.15735 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe7868e740 1 -- 172.21.15.19:0/2045969998 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dbc147a20 con 0x560dbc1ec4f0 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe6effd640 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560dbc1ec4f0 0x560dbc1ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe4effd640 10 client.15735 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.896+0000 7fbe6effd640 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560dbc1ec4f0 0x560dbc1ee8e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f616943cb0791ae server_cookie=137fb085fa3dee63 in_seq=0 out_seq=0 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe4effd640 1 -- 172.21.15.19:0/2045969998 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560dbc147a20 con 0x560dbc1ec4f0 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe4effd640 10 client.15735 handle_command_reply: tid=0 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe7868e740 1 client.15735 shutdown 2024-09-19T04:09:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe7868e740 1 -- 172.21.15.19:0/2045969998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d72e0 msgr2=0x560dbc1e11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d72e0 0x560dbc1e11b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800cfd0 tx=0x7fbe5809a760 comp rx=0 tx=0).stop 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe7868e740 1 -- 172.21.15.19:0/2045969998 shutdown_connections 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1dc370 0x560dbc1e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1d72e0 0x560dbc1e11b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.897+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560dbc1ec4f0 0x560dbc1ee8e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.898+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2045969998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.898+0000 7fbe4effd640 0 client.15735 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.898+0000 7fbe7868e740 1 -- 172.21.15.19:0/2045969998 >> 172.21.15.19:0/2045969998 conn(0x560dbc1d2010 msgr2=0x560dbc1d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.898+0000 7fbe7868e740 1 -- 172.21.15.19:0/2045969998 shutdown_connections 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.898+0000 7fbe7868e740 1 -- 172.21.15.19:0/2045969998 wait complete. 2024-09-19T04:09:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.898+0000 7fbe7868e740 20 client.15735 trim_cache size 0 max 16384 2024-09-19T04:09:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe7868e740 1 Processor -- start 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe7868e740 1 -- start start 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1f3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe7868e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701ac910 con 0x560dbc1dc370 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe7868e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701ae450 con 0x560dbc1d72e0 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe7868e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701acfe0 con 0x560dbc1d69a0 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45598/0 (socket says 172.21.15.19:45598) 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/2209655112 learned_addr learned my addr 172.21.15.19:0/2209655112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe6e7fc640 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.913+0000 7fbe6effd640 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/2209655112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 msgr2=0x560dbc1f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/2209655112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 msgr2=0x560dbc1e81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1e81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/2209655112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbc1d2ca0 con 0x560dbc1dc370 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1f3640 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe6002aeb0 tx=0x7fbe600098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0095d036d511365 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe657fa640 1 -- 172.21.15.19:0/2209655112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6004a070 con 0x560dbc1dc370 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe657fa640 1 -- 172.21.15.19:0/2209655112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe60036040 con 0x560dbc1dc370 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.914+0000 7fbe657fa640 1 -- 172.21.15.19:0/2209655112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600043c0 con 0x560dbc1dc370 2024-09-19T04:09:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 -- 172.21.15.19:0/2209655112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 msgr2=0x560dbc1f3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1f3640 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe6002aeb0 tx=0x7fbe600098b0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 -- 172.21.15.19:0/2209655112 shutdown_connections 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1f3640 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 --2- 172.21.15.19:0/2209655112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 -- 172.21.15.19:0/2209655112 >> 172.21.15.19:0/2209655112 conn(0x560dbc1d2010 msgr2=0x560dbc1d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 -- 172.21.15.19:0/2209655112 shutdown_connections 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 -- 172.21.15.19:0/2209655112 wait complete. 2024-09-19T04:09:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.915+0000 7fbe7868e740 1 -- start start 2024-09-19T04:09:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe7868e740 1 Processor -- start 2024-09-19T04:09:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1e6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe7868e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe7868e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701add80 con 0x560dbc1dc370 2024-09-19T04:09:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe7868e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701ad6b0 con 0x560dbc1d72e0 2024-09-19T04:09:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe7868e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701aa8f0 con 0x560dbc1d69a0 2024-09-19T04:09:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe7868e740 10 client.0 _ll_register_callbacks cb 0x7fbe580a0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1e68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45604/0 (socket says 172.21.15.19:45604) 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/3384104091 learned_addr learned my addr 172.21.15.19:0/3384104091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/3384104091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 msgr2=0x560dbc1e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/3384104091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 msgr2=0x560dbc1e6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.916+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/3384104091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbc1e6e10 con 0x560dbc1dc370 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1e68d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009430 tx=0x7fbe60031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4a1c00e5c1ee17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe4effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe4effd640 1 -- 172.21.15.19:0/3384104091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6004a070 con 0x560dbc1dc370 2024-09-19T04:09:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe7868e740 1 -- 172.21.15.19:0/3384104091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe60036040 con 0x560dbc1dc370 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe4effd640 1 -- 172.21.15.19:0/3384104091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fbe60004140 con 0x560dbc1dc370 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe4effd640 1 -- 172.21.15.19:0/3384104091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe60003cc0 con 0x560dbc1dc370 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe7868e740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe7868e740 1 -- 172.21.15.19:0/3384104091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560dbc257dc0 con 0x560dbc1dc370 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe4effd640 1 -- 172.21.15.19:0/3384104091 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe60036040 con 0x560dbc1dc370 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.917+0000 7fbe4effd640 1 -- 172.21.15.19:0/3384104091 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbe6002f730 con 0x560dbc1dc370 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe7868e740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe7868e740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe7868e740 20 client.15741 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe7868e740 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560dbc258770 0x560dbc25abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe7868e740 4 client.15741 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:09:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe7868e740 1 -- 172.21.15.19:0/3384104091 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dbc141d70 con 0x560dbc258770 2024-09-19T04:09:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe6effd640 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560dbc258770 0x560dbc25abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe6effd640 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560dbc258770 0x560dbc25abd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6afc0963e5aa98e4 server_cookie=b83f49fda56bc8 in_seq=0 out_seq=0 2024-09-19T04:09:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.918+0000 7fbe4effd640 10 client.15741 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe4effd640 1 -- 172.21.15.19:0/3384104091 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560dbc141d70 con 0x560dbc258770 2024-09-19T04:09:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe4effd640 10 client.15741 handle_command_reply: tid=0 2024-09-19T04:09:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 client.15741 shutdown 2024-09-19T04:09:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 -- 172.21.15.19:0/3384104091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 msgr2=0x560dbc1e68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1e68d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009430 tx=0x7fbe60031680 comp rx=0 tx=0).stop 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 -- 172.21.15.19:0/3384104091 shutdown_connections 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560dbc1dc370 0x560dbc1e68d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560dbc1d72e0 0x560dbc1e6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x560dbc258770 0x560dbc25abd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 --2- 172.21.15.19:0/3384104091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dbc1d69a0 0x560dbc1e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe7868e740 1 -- 172.21.15.19:0/3384104091 >> 172.21.15.19:0/3384104091 conn(0x560dbc1d2010 msgr2=0x560dbc1d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.943+0000 7fbe4effd640 0 client.15741 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7868e740 1 -- 172.21.15.19:0/3384104091 shutdown_connections 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7868e740 1 -- 172.21.15.19:0/3384104091 wait complete. 2024-09-19T04:09:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7868e740 20 client.15741 trim_cache size 0 max 16384 2024-09-19T04:09:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:09:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:09:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:09:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:09:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 -- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe5c0a3310 msgr2=0x7fbe5c0a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe5c0a3310 0x7fbe5c0a57d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe68005fd0 tx=0x7fbe6800b410 comp rx=0 tx=0).stop 2024-09-19T04:09:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 -- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe701013c0 msgr2=0x7fbe7019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe701013c0 0x7fbe7019a4b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800e9f0 tx=0x7fbe5800eeb0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 -- 172.21.15.19:0/825362805 shutdown_connections 2024-09-19T04:09:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fbe5c0a3310 0x7fbe5c0a57d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe70101d80 0x7fbe701a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe701013c0 0x7fbe7019a4b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 --2- 172.21.15.19:0/825362805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe701001d0 0x7fbe70199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 -- 172.21.15.19:0/825362805 >> 172.21.15.19:0/825362805 conn(0x7fbe700fb930 msgr2=0x7fbe701046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 -- 172.21.15.19:0/825362805 shutdown_connections 2024-09-19T04:09:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:15.944+0000 7fbe7581e640 1 -- 172.21.15.19:0/825362805 wait complete. 2024-09-19T04:09:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:16 smithi019 ceph-mon[30938]: pgmap v430: 129 pgs: 129 active+clean; 93 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 395 KiB/s wr, 116 op/s 2024-09-19T04:09:16.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:16 smithi196 ceph-mon[35731]: pgmap v430: 129 pgs: 129 active+clean; 93 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 395 KiB/s wr, 116 op/s 2024-09-19T04:09:16.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:16 smithi111 ceph-mon[35365]: pgmap v430: 129 pgs: 129 active+clean; 93 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 395 KiB/s wr, 116 op/s 2024-09-19T04:09:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-09-19T04:09:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-19T04:09:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-09-19T04:09:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-19T04:09:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-09-19T04:09:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-19T04:09:16.851 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-19T04:09:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-09-19T04:09:16.853 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-09-19T04:09:16.854 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-09-19T04:09:16.855 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-19T04:09:16.856 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-09-19T04:09:16.857 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-19T04:09:17.240 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-09-19T04:09:17.240 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-19T04:09:17.240 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-09-19T04:09:17.240 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-19T04:09:17.241 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-09-19T04:09:17.242 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-19T04:09:17.243 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-19T04:09:17.244 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-09-19T04:09:17.245 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-09-19T04:09:17.246 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-19T04:09:17.247 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-09-19T04:09:17.247 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-19T04:09:17.247 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-09-19T04:09:17.247 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-19T04:09:17.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-09-19T04:09:17.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-19T04:09:17.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-09-19T04:09:17.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-19T04:09:17.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-09-19T04:09:17.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-19T04:09:17.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-19T04:09:17.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-19T04:09:17.715 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-09-19T04:09:17.716 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-19T04:09:17.717 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-09-19T04:09:17.718 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-19T04:09:17.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-09-19T04:09:18.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:18 smithi019 ceph-mon[30938]: pgmap v431: 129 pgs: 129 active+clean; 222 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 773 KiB/s wr, 188 op/s 2024-09-19T04:09:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:18 smithi196 ceph-mon[35731]: pgmap v431: 129 pgs: 129 active+clean; 222 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 773 KiB/s wr, 188 op/s 2024-09-19T04:09:18.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:18 smithi111 ceph-mon[35365]: pgmap v431: 129 pgs: 129 active+clean; 222 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 773 KiB/s wr, 188 op/s 2024-09-19T04:09:18.771 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-19T04:09:18.771 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-09-19T04:09:18.771 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-19T04:09:18.772 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-19T04:09:18.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-19T04:09:18.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-09-19T04:09:18.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-19T04:09:18.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-09-19T04:09:18.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-19T04:09:18.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-09-19T04:09:19.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-19T04:09:19.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-09-19T04:09:19.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-19T04:09:19.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-09-19T04:09:19.719 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-19T04:09:19.720 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-19T04:09:19.721 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-09-19T04:09:19.722 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-19T04:09:19.723 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-09-19T04:09:19.724 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-09-19T04:09:19.725 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-19T04:09:20.076 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-09-19T04:09:20.077 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-19T04:09:20.078 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-19T04:09:20.079 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-19T04:09:20.080 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-09-19T04:09:20.081 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-19T04:09:20.082 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-09-19T04:09:20.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:20 smithi019 ceph-mon[30938]: pgmap v432: 129 pgs: 129 active+clean; 285 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 1.1 MiB/s wr, 273 op/s 2024-09-19T04:09:20.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:20 smithi196 ceph-mon[35731]: pgmap v432: 129 pgs: 129 active+clean; 285 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 1.1 MiB/s wr, 273 op/s 2024-09-19T04:09:20.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:20 smithi111 ceph-mon[35365]: pgmap v432: 129 pgs: 129 active+clean; 285 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 1.1 MiB/s wr, 273 op/s 2024-09-19T04:09:20.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-19T04:09:20.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-09-19T04:09:20.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-19T04:09:20.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-09-19T04:09:20.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-19T04:09:20.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-09-19T04:09:20.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-19T04:09:20.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-19T04:09:20.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-19T04:09:20.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-19T04:09:20.780 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-19T04:09:20.781 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-09-19T04:09:20.782 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-19T04:09:20.782 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-09-19T04:09:20.782 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-19T04:09:20.782 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-09-19T04:09:20.782 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-19T04:09:21.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-09-19T04:09:21.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-19T04:09:21.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-09-19T04:09:21.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-19T04:09:21.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-09-19T04:09:21.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-19T04:09:21.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-09-19T04:09:21.795 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-09-19T04:09:21.796 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-19T04:09:21.797 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-09-19T04:09:21.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-19T04:09:21.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-09-19T04:09:21.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-19T04:09:21.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-09-19T04:09:21.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-19T04:09:21.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-09-19T04:09:21.798 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-19T04:09:21.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-09-19T04:09:21.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-19T04:09:21.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-09-19T04:09:21.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-19T04:09:21.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-09-19T04:09:21.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-19T04:09:21.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-09-19T04:09:21.799 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-19T04:09:21.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-09-19T04:09:21.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-19T04:09:21.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-09-19T04:09:21.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-19T04:09:21.800 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-19T04:09:21.801 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-19T04:09:21.802 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-09-19T04:09:21.803 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-19T04:09:21.803 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-09-19T04:09:21.803 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-19T04:09:21.803 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-09-19T04:09:21.803 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-19T04:09:21.803 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-09-19T04:09:22.410 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-19T04:09:22.410 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-09-19T04:09:22.411 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-09-19T04:09:22.412 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-19T04:09:22.413 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-09-19T04:09:22.414 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-09-19T04:09:22.415 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-19T04:09:22.416 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-09-19T04:09:22.417 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-19T04:09:22.417 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-09-19T04:09:22.417 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-19T04:09:22.417 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-09-19T04:09:22.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:22 smithi019 ceph-mon[30938]: pgmap v433: 129 pgs: 129 active+clean; 213 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 1.1 MiB/s wr, 319 op/s 2024-09-19T04:09:22.456 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:22 smithi111 ceph-mon[35365]: pgmap v433: 129 pgs: 129 active+clean; 213 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 1.1 MiB/s wr, 319 op/s 2024-09-19T04:09:22.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:22 smithi196 ceph-mon[35731]: pgmap v433: 129 pgs: 129 active+clean; 213 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 1.1 MiB/s wr, 319 op/s 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-09-19T04:09:22.773 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-19T04:09:22.774 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-09-19T04:09:22.775 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-19T04:09:22.776 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-19T04:09:22.777 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-19T04:09:22.778 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-09-19T04:09:22.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-19T04:09:22.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-09-19T04:09:22.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-19T04:09:22.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-09-19T04:09:22.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-19T04:09:22.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-09-19T04:09:22.779 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-19T04:09:23.581 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-09-19T04:09:23.581 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-19T04:09:23.581 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-09-19T04:09:23.581 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-19T04:09:23.581 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-09-19T04:09:23.581 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-19T04:09:23.581 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-19T04:09:23.582 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-09-19T04:09:23.583 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-09-19T04:09:23.584 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-09-19T04:09:23.585 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-09-19T04:09:23.586 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-19T04:09:23.587 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-09-19T04:09:24.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:24 smithi019 ceph-mon[30938]: pgmap v434: 129 pgs: 129 active+clean; 209 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-09-19T04:09:24.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:24 smithi196 ceph-mon[35731]: pgmap v434: 129 pgs: 129 active+clean; 209 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-09-19T04:09:24.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:24 smithi111 ceph-mon[35365]: pgmap v434: 129 pgs: 129 active+clean; 209 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-09-19T04:09:24.707 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-19T04:09:24.708 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-09-19T04:09:24.709 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-19T04:09:24.710 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-19T04:09:24.711 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-09-19T04:09:24.712 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-19T04:09:24.713 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-09-19T04:09:24.714 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-19T04:09:25.115 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-09-19T04:09:25.115 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-19T04:09:25.115 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-09-19T04:09:25.115 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-19T04:09:25.115 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-09-19T04:09:25.115 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-19T04:09:25.116 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-19T04:09:25.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-09-19T04:09:25.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-19T04:09:25.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-09-19T04:09:25.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-19T04:09:25.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-09-19T04:09:25.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-19T04:09:25.487 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-19T04:09:25.488 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-19T04:09:25.489 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-09-19T04:09:25.490 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-09-19T04:09:25.491 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-19T04:09:25.492 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-19T04:09:25.493 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-09-19T04:09:25.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-19T04:09:25.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-09-19T04:09:25.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-19T04:09:25.494 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-09-19T04:09:26.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:26 smithi019 ceph-mon[30938]: pgmap v435: 129 pgs: 129 active+clean; 209 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-09-19T04:09:26.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:26 smithi196 ceph-mon[35731]: pgmap v435: 129 pgs: 129 active+clean; 209 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-09-19T04:09:26.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-19T04:09:26.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-09-19T04:09:26.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-19T04:09:26.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-19T04:09:26.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-19T04:09:26.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-19T04:09:26.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-09-19T04:09:26.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-19T04:09:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-09-19T04:09:26.476 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-19T04:09:26.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:26 smithi111 ceph-mon[35365]: pgmap v435: 129 pgs: 129 active+clean; 209 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-09-19T04:09:27.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-09-19T04:09:27.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-09-19T04:09:27.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-19T04:09:27.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-09-19T04:09:27.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-19T04:09:27.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-09-19T04:09:27.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-19T04:09:27.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-09-19T04:09:27.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-19T04:09:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-09-19T04:09:27.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-19T04:09:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-09-19T04:09:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-19T04:09:27.442 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-19T04:09:27.818 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-19T04:09:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-09-19T04:09:27.820 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-19T04:09:27.821 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-19T04:09:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-09-19T04:09:27.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-19T04:09:27.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-09-19T04:09:27.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-19T04:09:27.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-09-19T04:09:27.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-19T04:09:27.823 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-09-19T04:09:27.824 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-19T04:09:27.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-09-19T04:09:27.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-19T04:09:27.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-09-19T04:09:27.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-19T04:09:27.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-09-19T04:09:27.825 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-09-19T04:09:28.352 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-19T04:09:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-09-19T04:09:28.354 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-09-19T04:09:28.355 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-09-19T04:09:28.356 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-19T04:09:28.357 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-09-19T04:09:28.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-19T04:09:28.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-09-19T04:09:28.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-19T04:09:28.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-09-19T04:09:28.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-19T04:09:28.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-09-19T04:09:28.358 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-19T04:09:28.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:28 smithi019 ceph-mon[30938]: pgmap v436: 129 pgs: 129 active+clean; 260 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-19T04:09:28.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:28 smithi196 ceph-mon[35731]: pgmap v436: 129 pgs: 129 active+clean; 260 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-19T04:09:28.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:28 smithi111 ceph-mon[35365]: pgmap v436: 129 pgs: 129 active+clean; 260 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-19T04:09:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 -- 172.21.15.19:0/2710598378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 msgr2=0x7f6de4109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2710598378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 0x7f6de4109fa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009ca0 tx=0x7f6ddc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 -- 172.21.15.19:0/2710598378 shutdown_connections 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2710598378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 0x7f6de4109fa0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2710598378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de4102d50 0x7f6de41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2710598378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6de4101b60 0x7f6de4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 -- 172.21.15.19:0/2710598378 >> 172.21.15.19:0/2710598378 conn(0x7f6de40fd2c0 msgr2=0x7f6de40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 -- 172.21.15.19:0/2710598378 shutdown_connections 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.042+0000 7f6deb21e640 1 -- 172.21.15.19:0/2710598378 wait complete. 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6deb21e640 1 Processor -- start 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6deb21e640 1 -- start start 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6deb21e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6de4101b60 0x7f6de4075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6deb21e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de4102d50 0x7f6de40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6deb21e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 0x7f6de4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6deb21e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d7c0 con 0x7f6de4103710 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6deb21e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d640 con 0x7f6de4101b60 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6deb21e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d940 con 0x7f6de4102d50 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6de97bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 0x7f6de4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6de97bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 0x7f6de4074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43742/0 (socket says 172.21.15.19:43742) 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6de97bb640 1 -- 172.21.15.19:0/2660332429 learned_addr learned my addr 172.21.15.19:0/2660332429 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6de8fba640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6de4101b60 0x7f6de4075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.043+0000 7f6de3fff640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de4102d50 0x7f6de40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de97bb640 1 -- 172.21.15.19:0/2660332429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de4102d50 msgr2=0x7f6de40745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de97bb640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de4102d50 0x7f6de40745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de97bb640 1 -- 172.21.15.19:0/2660332429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6de4101b60 msgr2=0x7f6de4075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de97bb640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6de4101b60 0x7f6de4075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de97bb640 1 -- 172.21.15.19:0/2660332429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de40753b0 con 0x7f6de4103710 2024-09-19T04:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de8fba640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6de4101b60 0x7f6de4075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de97bb640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 0x7f6de4074b20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc005e30 tx=0x7f6ddc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de1ffb640 1 -- 172.21.15.19:0/2660332429 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc003b70 con 0x7f6de4103710 2024-09-19T04:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6deb21e640 1 -- 172.21.15.19:0/2660332429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de41a1ed0 con 0x7f6de4103710 2024-09-19T04:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de1ffb640 1 -- 172.21.15.19:0/2660332429 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc003d10 con 0x7f6de4103710 2024-09-19T04:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6de1ffb640 1 -- 172.21.15.19:0/2660332429 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc02fad0 con 0x7f6de4103710 2024-09-19T04:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.044+0000 7f6deb21e640 1 -- 172.21.15.19:0/2660332429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de41a2340 con 0x7f6de4103710 2024-09-19T04:09:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6de1ffb640 1 -- 172.21.15.19:0/2660332429 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6ddc0045c0 con 0x7f6de4103710 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6dee08d740 1 Processor -- start 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6dee08d740 1 -- start start 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 0x5645ca893da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8942e0 0x5645ca898da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca89b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6dee08d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5645ca7f72f0 con 0x5645ca8942e0 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6dee08d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5645ca79f270 con 0x5645ca899370 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6dee08d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5645ca7f32b0 con 0x5645ca8939a0 2024-09-19T04:09:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6de1ffb640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6dd40a32f0 0x7f6dd40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6de8fba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8942e0 0x5645ca898da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6de8fba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8942e0 0x5645ca898da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43758/0 (socket says 172.21.15.19:43758) 2024-09-19T04:09:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6de8fba640 1 -- 172.21.15.19:0/616299140 learned_addr learned my addr 172.21.15.19:0/616299140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.047+0000 7f6de3fff640 1 --2- 172.21.15.19:0/616299140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 0x5645ca893da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de97bb640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6dd40a32f0 0x7f6dd40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de1ffb640 1 -- 172.21.15.19:0/2660332429 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6ddc0f0c80 con 0x7f6de4103710 2024-09-19T04:09:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de8fba640 1 -- 172.21.15.19:0/616299140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 msgr2=0x5645ca893da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de8fba640 1 --2- 172.21.15.19:0/616299140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 0x5645ca893da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de8fba640 1 -- 172.21.15.19:0/616299140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 msgr2=0x5645ca89b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de8fba640 1 --2- 172.21.15.19:0/616299140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca89b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de8fba640 1 -- 172.21.15.19:0/616299140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ca79e340 con 0x5645ca8942e0 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de8fba640 1 --2- 172.21.15.19:0/616299140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8942e0 0x5645ca898da0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80114c0 tx=0x7f6dd8011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bf52bca4a6b6c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6de97bb640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6dd40a32f0 0x7f6dd40a57b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc04c000 tx=0x7f6ddc02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6dceffd640 1 -- 172.21.15.19:0/616299140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8004020 con 0x5645ca8942e0 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6dceffd640 1 -- 172.21.15.19:0/616299140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd80041c0 con 0x5645ca8942e0 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.048+0000 7f6dceffd640 1 -- 172.21.15.19:0/616299140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800f930 con 0x5645ca8942e0 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 -- 172.21.15.19:0/616299140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8942e0 msgr2=0x5645ca898da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 --2- 172.21.15.19:0/616299140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8942e0 0x5645ca898da0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80114c0 tx=0x7f6dd8011980 comp rx=0 tx=0).stop 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 -- 172.21.15.19:0/616299140 shutdown_connections 2024-09-19T04:09:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 --2- 172.21.15.19:0/616299140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca89b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 --2- 172.21.15.19:0/616299140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8942e0 0x5645ca898da0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 --2- 172.21.15.19:0/616299140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 0x5645ca893da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 -- 172.21.15.19:0/616299140 >> 172.21.15.19:0/616299140 conn(0x5645ca88f010 msgr2=0x5645ca88f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 -- 172.21.15.19:0/616299140 shutdown_connections 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.049+0000 7f6dee08d740 1 -- 172.21.15.19:0/616299140 wait complete. 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 1 -- start start 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 1 Processor -- start 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca89db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca89e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca8a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5645ca89c340 con 0x5645ca8939a0 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5645ca780290 con 0x5645ca899370 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5645ca89c4c0 con 0x5645ca8942e0 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6dee08d740 10 client.0 _ll_register_callbacks cb 0x5645ca7b1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6de3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca89db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6de8fba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca89e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6de3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca89db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43766/0 (socket says 172.21.15.19:43766) 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.050+0000 7f6de3fff640 1 -- 172.21.15.19:0/2011254480 learned_addr learned my addr 172.21.15.19:0/2011254480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6de3fff640 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca8a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6de3fff640 1 -- 172.21.15.19:0/2011254480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 msgr2=0x5645ca89e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6de3fff640 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca89e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6de3fff640 1 -- 172.21.15.19:0/2011254480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 msgr2=0x5645ca8a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6de3fff640 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca8a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6de3fff640 1 -- 172.21.15.19:0/2011254480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ca50a210 con 0x5645ca8939a0 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6de8fba640 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca89e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6de3fff640 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca89db60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0012d30 tx=0x7f6dd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27e168cfdff5fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6dc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6dc3fff640 1 -- 172.21.15.19:0/2011254480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0004030 con 0x5645ca8939a0 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6dc3fff640 1 -- 172.21.15.19:0/2011254480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6dd00041d0 con 0x5645ca8939a0 2024-09-19T04:09:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6dee08d740 1 -- 172.21.15.19:0/2011254480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd80041c0 con 0x5645ca8939a0 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6dc3fff640 1 -- 172.21.15.19:0/2011254480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6dd0026050 con 0x5645ca8939a0 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6dee08d740 10 client.15753 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.051+0000 7f6dee08d740 1 -- 172.21.15.19:0/2011254480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645ca5d6620 con 0x5645ca8939a0 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6dc3fff640 1 -- 172.21.15.19:0/2011254480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd80041c0 con 0x5645ca8939a0 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6dc3fff640 1 -- 172.21.15.19:0/2011254480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6dd0010430 con 0x5645ca8939a0 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6dee08d740 10 client.15753 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6dee08d740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6dee08d740 20 client.15753 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6dee08d740 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5645ca8a9830 0x5645ca8abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6dee08d740 4 client.15753 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6dee08d740 1 -- 172.21.15.19:0/2011254480 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645ca804a20 con 0x5645ca8a9830 2024-09-19T04:09:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.052+0000 7f6de8fba640 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5645ca8a9830 0x5645ca8abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.053+0000 7f6dc3fff640 10 client.15753 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.053+0000 7f6de8fba640 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5645ca8a9830 0x5645ca8abca0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482945157a494955 server_cookie=ba730e7f45b03bcc in_seq=0 out_seq=0 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.053+0000 7f6dc3fff640 1 -- 172.21.15.19:0/2011254480 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645ca804a20 con 0x5645ca8a9830 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.053+0000 7f6dc3fff640 10 client.15753 handle_command_reply: tid=0 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 client.15753 shutdown 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 -- 172.21.15.19:0/2011254480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 msgr2=0x5645ca89db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca89db60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0012d30 tx=0x7f6dd0009bb0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 -- 172.21.15.19:0/2011254480 shutdown_connections 2024-09-19T04:09:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca8a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5645ca8a9830 0x5645ca8abca0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca89e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 --2- 172.21.15.19:0/2011254480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca89db60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dc3fff640 0 client.15753 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 -- 172.21.15.19:0/2011254480 >> 172.21.15.19:0/2011254480 conn(0x5645ca88f010 msgr2=0x5645ca890820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 -- 172.21.15.19:0/2011254480 shutdown_connections 2024-09-19T04:09:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 1 -- 172.21.15.19:0/2011254480 wait complete. 2024-09-19T04:09:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.054+0000 7f6dee08d740 20 client.15753 trim_cache size 0 max 16384 2024-09-19T04:09:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6dee08d740 1 Processor -- start 2024-09-19T04:09:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6dee08d740 1 -- start start 2024-09-19T04:09:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 0x5645ca89e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca8942e0 0x5645ca8a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca899370 0x5645ca88e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6dee08d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41ac990 con 0x5645ca899370 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6dee08d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41ae4d0 con 0x5645ca8942e0 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6dee08d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41ad060 con 0x5645ca8939a0 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca899370 0x5645ca88e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de8fba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca8942e0 0x5645ca8a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de8fba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca8942e0 0x5645ca8a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.19:38590/0 (socket says 172.21.15.19:38590) 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca899370 0x5645ca88e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43782/0 (socket says 172.21.15.19:43782) 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de8fba640 1 -- 172.21.15.19:0/3909603054 learned_addr learned my addr 172.21.15.19:0/3909603054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de3fff640 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 0x5645ca89e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de8fba640 1 -- 172.21.15.19:0/3909603054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 msgr2=0x5645ca89e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de8fba640 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 0x5645ca89e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de8fba640 1 -- 172.21.15.19:0/3909603054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca899370 msgr2=0x5645ca88e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de8fba640 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca899370 0x5645ca88e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de8fba640 1 -- 172.21.15.19:0/3909603054 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ca89d6a0 con 0x5645ca8942e0 2024-09-19T04:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.069+0000 7f6de3fff640 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca899370 0x5645ca88e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6de8fba640 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca8942e0 0x5645ca8a50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80058c0 tx=0x7f6dd8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56de1bf4c48456e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dce7fc640 1 -- 172.21.15.19:0/3909603054 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801d040 con 0x5645ca8942e0 2024-09-19T04:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dce7fc640 1 -- 172.21.15.19:0/3909603054 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd802a070 con 0x5645ca8942e0 2024-09-19T04:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dce7fc640 1 -- 172.21.15.19:0/3909603054 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8005500 con 0x5645ca8942e0 2024-09-19T04:09:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dee08d740 1 -- 172.21.15.19:0/3909603054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca8942e0 msgr2=0x5645ca8a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dee08d740 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca8942e0 0x5645ca8a50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80058c0 tx=0x7f6dd8009310 comp rx=0 tx=0).stop 2024-09-19T04:09:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dee08d740 1 -- 172.21.15.19:0/3909603054 shutdown_connections 2024-09-19T04:09:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dee08d740 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca899370 0x5645ca88e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dee08d740 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca8942e0 0x5645ca8a50a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dee08d740 1 --2- 172.21.15.19:0/3909603054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8939a0 0x5645ca89e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.070+0000 7f6dee08d740 1 -- 172.21.15.19:0/3909603054 >> 172.21.15.19:0/3909603054 conn(0x5645ca88f010 msgr2=0x5645ca88d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 -- 172.21.15.19:0/3909603054 shutdown_connections 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 -- 172.21.15.19:0/3909603054 wait complete. 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 -- start start 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 Processor -- start 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca8a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca8a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca8a4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41ade00 con 0x5645ca8939a0 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41ad730 con 0x5645ca899370 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41aa970 con 0x5645ca8942e0 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6dee08d740 10 client.0 _ll_register_callbacks cb 0x7f6dd0009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6de3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca8a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6de3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca8a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43794/0 (socket says 172.21.15.19:43794) 2024-09-19T04:09:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6de3fff640 1 -- 172.21.15.19:0/536730472 learned_addr learned my addr 172.21.15.19:0/536730472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.071+0000 7f6de8fba640 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca8a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6de3fff640 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca8a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6de3fff640 1 -- 172.21.15.19:0/536730472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 msgr2=0x5645ca8a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6de3fff640 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca8a3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6de3fff640 1 -- 172.21.15.19:0/536730472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 msgr2=0x5645ca8a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6de3fff640 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca8a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6de3fff640 1 -- 172.21.15.19:0/536730472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ca894f70 con 0x5645ca8939a0 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6de3fff640 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca8a2c00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0010800 tx=0x7f6dd00059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321e34bef1b9b2eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6dc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6dc3fff640 1 -- 172.21.15.19:0/536730472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0004460 con 0x5645ca8939a0 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6dc3fff640 1 -- 172.21.15.19:0/536730472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f6dd0010a20 con 0x5645ca8939a0 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6dee08d740 1 -- 172.21.15.19:0/536730472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd802a070 con 0x5645ca8939a0 2024-09-19T04:09:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6dc3fff640 1 -- 172.21.15.19:0/536730472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6dd0010c20 con 0x5645ca8939a0 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6dee08d740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6dee08d740 1 -- 172.21.15.19:0/536730472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645ca88fca0 con 0x5645ca8939a0 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.072+0000 7f6dc3fff640 1 -- 172.21.15.19:0/536730472 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd802a070 con 0x5645ca8939a0 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.073+0000 7f6dc3fff640 1 -- 172.21.15.19:0/536730472 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6dd00047e0 con 0x5645ca8939a0 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.073+0000 7f6dee08d740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.073+0000 7f6dee08d740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.073+0000 7f6dee08d740 20 client.15759 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.073+0000 7f6dee08d740 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5645ca914f90 0x5645ca9173b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.073+0000 7f6dee08d740 4 client.15759 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.073+0000 7f6dee08d740 1 -- 172.21.15.19:0/536730472 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645ca7fed70 con 0x5645ca914f90 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.073+0000 7f6de8fba640 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5645ca914f90 0x5645ca9173b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.075+0000 7f6dc3fff640 10 client.15759 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.075+0000 7f6de8fba640 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5645ca914f90 0x5645ca9173b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc2c467a4d0195f server_cookie=1629fe9fe56fd2a3 in_seq=0 out_seq=0 2024-09-19T04:09:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dc3fff640 1 -- 172.21.15.19:0/536730472 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645ca7fed70 con 0x5645ca914f90 2024-09-19T04:09:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dc3fff640 10 client.15759 handle_command_reply: tid=0 2024-09-19T04:09:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 client.15759 shutdown 2024-09-19T04:09:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 -- 172.21.15.19:0/536730472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 msgr2=0x5645ca8a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca8a2c00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0010800 tx=0x7f6dd00059b0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 -- 172.21.15.19:0/536730472 shutdown_connections 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5645ca899370 0x5645ca8a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5645ca914f90 0x5645ca9173b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645ca8942e0 0x5645ca8a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dc3fff640 0 client.15759 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 --2- 172.21.15.19:0/536730472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645ca8939a0 0x5645ca8a2c00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 -- 172.21.15.19:0/536730472 >> 172.21.15.19:0/536730472 conn(0x5645ca88f010 msgr2=0x5645ca88fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 -- 172.21.15.19:0/536730472 shutdown_connections 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.154+0000 7f6dee08d740 1 -- 172.21.15.19:0/536730472 wait complete. 2024-09-19T04:09:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6dee08d740 20 client.15759 trim_cache size 0 max 16384 2024-09-19T04:09:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-09-19T04:09:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-09-19T04:09:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-09-19T04:09:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-09-19T04:09:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 -- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6dd40a32f0 msgr2=0x7f6dd40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6dd40a32f0 0x7f6dd40a57b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc04c000 tx=0x7f6ddc02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 -- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 msgr2=0x7f6de4074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 0x7f6de4074b20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc005e30 tx=0x7f6ddc004ab0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 -- 172.21.15.19:0/2660332429 shutdown_connections 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f6dd40a32f0 0x7f6dd40a57b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6de4103710 0x7f6de4074b20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de4102d50 0x7f6de40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 --2- 172.21.15.19:0/2660332429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6de4101b60 0x7f6de4075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 -- 172.21.15.19:0/2660332429 >> 172.21.15.19:0/2660332429 conn(0x7f6de40fd2c0 msgr2=0x7f6de41060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 -- 172.21.15.19:0/2660332429 shutdown_connections 2024-09-19T04:09:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-09-19T04:09:29.155+0000 7f6deb21e640 1 -- 172.21.15.19:0/2660332429 wait complete. 2024-09-19T04:09:29.426 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-09-19T04:09:29.426 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-19T04:09:29.426 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-09-19T04:09:29.426 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-19T04:09:29.426 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-09-19T04:09:29.427 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-19T04:09:29.427 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-09-19T04:09:29.427 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-19T04:09:29.427 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-09-19T04:09:29.427 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-19T04:09:29.427 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-09-19T04:09:29.428 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-19T04:09:29.428 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-09-19T04:09:29.428 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-19T04:09:29.428 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-09-19T04:09:29.428 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-19T04:09:29.428 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-09-19T04:09:29.429 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-19T04:09:29.429 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-09-19T04:09:29.429 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-19T04:09:29.429 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-09-19T04:09:29.429 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-19T04:09:29.430 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-09-19T04:09:29.430 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-19T04:09:29.430 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-09-19T04:09:29.430 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-19T04:09:29.430 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-09-19T04:09:29.430 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-19T04:09:29.431 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-09-19T04:09:29.431 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-19T04:09:29.431 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-09-19T04:09:29.431 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-19T04:09:29.431 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-09-19T04:09:29.431 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-19T04:09:29.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-09-19T04:09:29.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-19T04:09:29.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-09-19T04:09:29.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-19T04:09:29.432 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-09-19T04:09:29.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-19T04:09:29.433 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-09-19T04:09:29.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-19T04:09:29.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-09-19T04:09:29.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-19T04:09:29.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-09-19T04:09:29.434 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-19T04:09:29.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-09-19T04:09:29.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-19T04:09:29.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-09-19T04:09:29.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-19T04:09:29.435 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-09-19T04:09:29.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-19T04:09:29.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-09-19T04:09:29.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-19T04:09:29.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-09-19T04:09:29.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-19T04:09:29.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-09-19T04:09:29.436 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-19T04:09:29.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-09-19T04:09:29.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-19T04:09:29.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-09-19T04:09:29.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-19T04:09:29.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-09-19T04:09:29.437 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-19T04:09:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-09-19T04:09:29.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-19T04:09:29.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-09-19T04:09:29.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-19T04:09:29.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-09-19T04:09:29.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-19T04:09:29.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-09-19T04:09:29.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-19T04:09:29.439 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-09-19T04:09:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-19T04:09:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-09-19T04:09:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-19T04:09:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-09-19T04:09:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-19T04:09:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-09-19T04:09:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-19T04:09:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-19T04:09:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-19T04:09:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-09-19T04:09:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-09-19T04:09:30.292 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-09-19T04:09:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-19T04:09:30.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-09-19T04:09:30.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-19T04:09:30.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-09-19T04:09:30.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-19T04:09:30.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-09-19T04:09:30.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-19T04:09:30.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-09-19T04:09:30.294 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-19T04:09:30.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:30 smithi196 ceph-mon[35731]: pgmap v437: 129 pgs: 129 active+clean; 324 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s rd, 995 KiB/s wr, 354 op/s 2024-09-19T04:09:30.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:30 smithi111 ceph-mon[35365]: pgmap v437: 129 pgs: 129 active+clean; 324 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s rd, 995 KiB/s wr, 354 op/s 2024-09-19T04:09:30.680 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-09-19T04:09:30.680 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-19T04:09:30.680 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-09-19T04:09:30.680 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-19T04:09:30.680 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-09-19T04:09:30.680 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-19T04:09:30.680 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-09-19T04:09:30.680 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-19T04:09:30.681 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-09-19T04:09:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-19T04:09:30.683 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-09-19T04:09:30.684 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-19T04:09:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-09-19T04:09:30.686 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-19T04:09:30.687 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-09-19T04:09:30.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:30 smithi019 ceph-mon[30938]: pgmap v437: 129 pgs: 129 active+clean; 324 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s rd, 995 KiB/s wr, 354 op/s 2024-09-19T04:09:31.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-19T04:09:31.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-09-19T04:09:31.449 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-09-19T04:09:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-09-19T04:09:31.451 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-09-19T04:09:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-09-19T04:09:31.453 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-09-19T04:09:31.454 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-09-19T04:09:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-19T04:09:31.456 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-09-19T04:09:31.456 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-19T04:09:32.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:32 smithi196 ceph-mon[35731]: pgmap v438: 129 pgs: 129 active+clean; 355 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s rd, 1001 KiB/s wr, 382 op/s 2024-09-19T04:09:32.468 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-19T04:09:32.469 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-09-19T04:09:32.470 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-19T04:09:32.471 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-09-19T04:09:32.472 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-09-19T04:09:32.473 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-19T04:09:32.474 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-09-19T04:09:32.475 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-19T04:09:32.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:32 smithi111 ceph-mon[35365]: pgmap v438: 129 pgs: 129 active+clean; 355 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s rd, 1001 KiB/s wr, 382 op/s 2024-09-19T04:09:32.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:32 smithi019 ceph-mon[30938]: pgmap v438: 129 pgs: 129 active+clean; 355 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s rd, 1001 KiB/s wr, 382 op/s 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-09-19T04:09:33.117 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-19T04:09:33.118 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-19T04:09:33.119 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-09-19T04:09:33.120 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-19T04:09:33.121 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-09-19T04:09:33.122 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-19T04:09:33.123 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-09-19T04:09:33.123 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-19T04:09:33.123 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-09-19T04:09:33.123 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-19T04:09:33.123 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-09-19T04:09:33.123 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-19T04:09:33.123 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-09-19T04:09:33.556 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-19T04:09:33.556 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-09-19T04:09:33.556 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-19T04:09:33.556 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-19T04:09:33.557 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-09-19T04:09:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-09-19T04:09:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-19T04:09:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-09-19T04:09:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-19T04:09:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-09-19T04:09:33.563 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-19T04:09:33.633 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-09-19T04:09:33.633 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-19T04:09:33.633 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-09-19T04:09:33.633 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-19T04:09:33.633 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-09-19T04:09:33.633 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-19T04:09:33.634 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-09-19T04:09:33.634 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-19T04:09:33.634 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-09-19T04:09:33.634 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-19T04:09:33.634 INFO:tasks.workunit.client.0.smithi019.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-09-19T04:09:33.634 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-19T04:09:33.634 INFO:tasks.workunit.client.0.smithi019.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-09-19T04:09:33.634 INFO:tasks.workunit.client.0.smithi019.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-19T04:09:33.634 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T04:09:33.634 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T04:09:33.667 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-09-19T04:09:33.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T04:09:34.016 DEBUG:teuthology.parallel:result is None 2024-09-19T04:09:34.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T04:09:34.051 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T04:09:34.051 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T04:09:34.061 INFO:tasks.quiescer:joining Quiescers 2024-09-19T04:09:34.061 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T04:09:34.061 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:09:34.349 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:34.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:34 smithi019 ceph-mon[30938]: pgmap v439: 129 pgs: 129 active+clean; 351 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 973 KiB/s wr, 369 op/s 2024-09-19T04:09:34.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:34 smithi196 ceph-mon[35731]: pgmap v439: 129 pgs: 129 active+clean; 351 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 973 KiB/s wr, 369 op/s 2024-09-19T04:09:34.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:34 smithi111 ceph-mon[35365]: pgmap v439: 129 pgs: 129 active+clean; 351 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 973 KiB/s wr, 369 op/s 2024-09-19T04:09:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 -- 172.21.15.19:0/1111506770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 msgr2=0x7f501c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 --2- 172.21.15.19:0/1111506770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 0x7f501c10da40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5014009ca0 tx=0x7f501402a950 comp rx=0 tx=0).stop 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 -- 172.21.15.19:0/1111506770 shutdown_connections 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 --2- 172.21.15.19:0/1111506770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 0x7f501c10da40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 --2- 172.21.15.19:0/1111506770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f501c102570 0x7f501c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 --2- 172.21.15.19:0/1111506770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f501c100e20 0x7f501c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 -- 172.21.15.19:0/1111506770 >> 172.21.15.19:0/1111506770 conn(0x7f501c0fc7d0 msgr2=0x7f501c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 -- 172.21.15.19:0/1111506770 shutdown_connections 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.890+0000 7f50208d0640 1 -- 172.21.15.19:0/1111506770 wait complete. 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.891+0000 7f50208d0640 1 Processor -- start 2024-09-19T04:09:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.891+0000 7f50208d0640 1 -- start start 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f50208d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f501c100e20 0x7f501c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f50208d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f501c102570 0x7f501c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f50208d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 0x7f501c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f50208d0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f501c110f80 con 0x7f501c102ef0 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f50208d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f501c110e00 con 0x7f501c102570 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f50208d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f501c111100 con 0x7f501c100e20 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 0x7f501c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 0x7f501c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45838/0 (socket says 172.21.15.19:45838) 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 -- 172.21.15.19:0/526007884 learned_addr learned my addr 172.21.15.19:0/526007884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501a59c640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f501c100e20 0x7f501c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f5013fff640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f501c102570 0x7f501c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 -- 172.21.15.19:0/526007884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f501c100e20 msgr2=0x7f501c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f501c100e20 0x7f501c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 -- 172.21.15.19:0/526007884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f501c102570 msgr2=0x7f501c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f501c102570 0x7f501c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 -- 172.21.15.19:0/526007884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c19d400 con 0x7f501c102ef0 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.892+0000 7f501ad9d640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 0x7f501c19cd20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5014004af0 tx=0x7f501402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.893+0000 7f50137fe640 1 -- 172.21.15.19:0/526007884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014034040 con 0x7f501c102ef0 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.893+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c19d690 con 0x7f501c102ef0 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.893+0000 7f50137fe640 1 -- 172.21.15.19:0/526007884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5014030920 con 0x7f501c102ef0 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.893+0000 7f50137fe640 1 -- 172.21.15.19:0/526007884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014004820 con 0x7f501c102ef0 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.893+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c1a4fd0 con 0x7f501c102ef0 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.894+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501c1021b0 con 0x7f501c102ef0 2024-09-19T04:09:34.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.896+0000 7f50137fe640 1 -- 172.21.15.19:0/526007884 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5014037020 con 0x7f501c102ef0 2024-09-19T04:09:34.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.896+0000 7f50137fe640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50080a3240 0x7f50080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:34.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.897+0000 7f50137fe640 1 -- 172.21.15.19:0/526007884 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f50140f0a80 con 0x7f501c102ef0 2024-09-19T04:09:34.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.897+0000 7f501a59c640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50080a3240 0x7f50080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:34.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.897+0000 7f501a59c640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50080a3240 0x7f50080a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f500c0058c0 tx=0x7f500c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:34.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:34.901+0000 7f50137fe640 1 -- 172.21.15.19:0/526007884 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f50140bd400 con 0x7f501c102ef0 2024-09-19T04:09:35.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.117+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f501c10a0a0 con 0x7f501c102ef0 2024-09-19T04:09:35.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.118+0000 7f50137fe640 1 -- 172.21.15.19:0/526007884 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7f50140bcd00 con 0x7f501c102ef0 2024-09-19T04:09:35.120 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:09:35.120 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:09:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50080a3240 msgr2=0x7f50080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50080a3240 0x7f50080a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f500c0058c0 tx=0x7f500c00a250 comp rx=0 tx=0).stop 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 msgr2=0x7f501c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 0x7f501c19cd20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5014004af0 tx=0x7f501402f5b0 comp rx=0 tx=0).stop 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 shutdown_connections 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f50080a3240 0x7f50080a5700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501c102ef0 0x7f501c19cd20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f501c102570 0x7f501c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 --2- 172.21.15.19:0/526007884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f501c100e20 0x7f501c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 >> 172.21.15.19:0/526007884 conn(0x7f501c0fc7d0 msgr2=0x7f501c06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 shutdown_connections 2024-09-19T04:09:35.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:35.120+0000 7f50208d0640 1 -- 172.21.15.19:0/526007884 wait complete. 2024-09-19T04:09:35.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:35 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/526007884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:09:35.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:35 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/526007884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:09:35.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:35 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/526007884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:09:35.874 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --cancel --all' 2024-09-19T04:09:35.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --cancel --all 2024-09-19T04:09:36.138 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:36.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:36 smithi196 ceph-mon[35731]: pgmap v440: 129 pgs: 129 active+clean; 229 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s rd, 1.2 MiB/s wr, 471 op/s 2024-09-19T04:09:36.516 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:36 smithi019 ceph-mon[30938]: pgmap v440: 129 pgs: 129 active+clean; 229 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s rd, 1.2 MiB/s wr, 471 op/s 2024-09-19T04:09:36.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 -- 172.21.15.19:0/1547225070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40102d30 msgr2=0x7fcc40103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 --2- 172.21.15.19:0/1547225070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40102d30 0x7fcc40103110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009930 tx=0x7fcc3002a440 comp rx=0 tx=0).stop 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 -- 172.21.15.19:0/1547225070 shutdown_connections 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 --2- 172.21.15.19:0/1547225070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc401082a0 0x7fcc4010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 --2- 172.21.15.19:0/1547225070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc401036e0 0x7fcc40107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 --2- 172.21.15.19:0/1547225070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40102d30 0x7fcc40103110 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 -- 172.21.15.19:0/1547225070 >> 172.21.15.19:0/1547225070 conn(0x7fcc400fc7d0 msgr2=0x7fcc400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 -- 172.21.15.19:0/1547225070 shutdown_connections 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.602+0000 7fcc44f08640 1 -- 172.21.15.19:0/1547225070 wait complete. 2024-09-19T04:09:36.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc44f08640 1 Processor -- start 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc44f08640 1 -- start start 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc44f08640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc40102d30 0x7fcc40198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc44f08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc401036e0 0x7fcc40198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc44f08640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc401082a0 0x7fcc4019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc44f08640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401120e0 con 0x7fcc401082a0 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc44f08640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40111f60 con 0x7fcc401036e0 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc44f08640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40112260 con 0x7fcc40102d30 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc3ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc401082a0 0x7fcc4019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc3ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc401082a0 0x7fcc4019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45858/0 (socket says 172.21.15.19:45858) 2024-09-19T04:09:36.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc3ed9d640 1 -- 172.21.15.19:0/4181383305 learned_addr learned my addr 172.21.15.19:0/4181383305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc401036e0 0x7fcc40198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.603+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc40102d30 0x7fcc40198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc3ed9d640 1 -- 172.21.15.19:0/4181383305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc40102d30 msgr2=0x7fcc40198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc3ed9d640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc40102d30 0x7fcc40198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc3ed9d640 1 -- 172.21.15.19:0/4181383305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc401036e0 msgr2=0x7fcc40198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc3ed9d640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc401036e0 0x7fcc40198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc3ed9d640 1 -- 172.21.15.19:0/4181383305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4019d600 con 0x7fcc401082a0 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc40102d30 0x7fcc40198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc3ed9d640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc401082a0 0x7fcc4019cf20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800cad0 tx=0x7fcc3800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc2f7fe640 1 -- 172.21.15.19:0/4181383305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800ad60 con 0x7fcc401082a0 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc2f7fe640 1 -- 172.21.15.19:0/4181383305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc38005020 con 0x7fcc401082a0 2024-09-19T04:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc44f08640 1 -- 172.21.15.19:0/4181383305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4019d8f0 con 0x7fcc401082a0 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc2f7fe640 1 -- 172.21.15.19:0/4181383305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800f6b0 con 0x7fcc401082a0 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.604+0000 7fcc44f08640 1 -- 172.21.15.19:0/4181383305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc401a51d0 con 0x7fcc401082a0 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc2f7fe640 1 -- 172.21.15.19:0/4181383305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcc38005560 con 0x7fcc401082a0 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc472e1740 1 Processor -- start 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc472e1740 1 -- start start 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc35cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 0x5626cdc44ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc472e1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5626cdb9a200 con 0x5626cdc44ff0 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc472e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5626cdb40dc0 con 0x5626cdc358e0 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc472e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626cdb961c0 con 0x5626cdc36200 2024-09-19T04:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc3dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc3dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45862/0 (socket says 172.21.15.19:45862) 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2414395457 learned_addr learned my addr 172.21.15.19:0/2414395457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 0x5626cdc44ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.607+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc35cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc2f7fe640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc280a3240 0x7fcc280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3e59c640 1 -- 172.21.15.19:0/2414395457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 msgr2=0x5626cdc35cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3ed9d640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc280a3240 0x7fcc280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc35cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3e59c640 1 -- 172.21.15.19:0/2414395457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 msgr2=0x5626cdc473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3e59c640 1 -- 172.21.15.19:0/2414395457 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626cd90b620 con 0x5626cdc36200 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc2f7fe640 1 -- 172.21.15.19:0/4181383305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fcc380cfd50 con 0x7fcc401082a0 2024-09-19T04:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 0x5626cdc44ab0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009900 tx=0x7fcc300316e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f90a7c17da8bf887 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc3ed9d640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc280a3240 0x7fcc280a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc3809bcd0 tx=0x7fcc3806ee50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2414395457 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3003f070 con 0x5626cdc36200 2024-09-19T04:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2414395457 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc300035f0 con 0x5626cdc36200 2024-09-19T04:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.608+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2414395457 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30035070 con 0x5626cdc36200 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 -- 172.21.15.19:0/2414395457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 msgr2=0x5626cdc44ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 0x5626cdc44ab0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009900 tx=0x7fcc300316e0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 -- 172.21.15.19:0/2414395457 shutdown_connections 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 0x5626cdc44ab0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2414395457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc35cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 -- 172.21.15.19:0/2414395457 >> 172.21.15.19:0/2414395457 conn(0x5626cdc31f90 msgr2=0x5626cdc32370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 -- 172.21.15.19:0/2414395457 shutdown_connections 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.609+0000 7fcc472e1740 1 -- 172.21.15.19:0/2414395457 wait complete. 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 1 -- start start 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 1 Processor -- start 2024-09-19T04:09:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc44940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc36200 0x5626cdc41bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc42150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5626cdc47bc0 con 0x5626cdc44ff0 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5626cdb34c50 con 0x5626cdc36200 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626cdc47d40 con 0x5626cdc358e0 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc472e1740 10 client.0 _ll_register_callbacks cb 0x5626cdb46ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc3dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc42150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc3dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc42150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45866/0 (socket says 172.21.15.19:45866) 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2550438518 learned_addr learned my addr 172.21.15.19:0/2550438518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc44940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.610+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc36200 0x5626cdc41bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2550438518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 msgr2=0x5626cdc44940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc44940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc36200 msgr2=0x5626cdc41bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc36200 0x5626cdc41bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2550438518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626cdc42980 con 0x5626cdc44ff0 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc36200 0x5626cdc41bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc42150 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009c40 tx=0x7fcc34008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80490b8f7609d377 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2550438518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34013c90 con 0x5626cdc44ff0 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2550438518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fcc3400f040 con 0x5626cdc44ff0 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc472e1740 1 -- 172.21.15.19:0/2550438518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc300035f0 con 0x5626cdc44ff0 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2550438518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc3402b050 con 0x5626cdc44ff0 2024-09-19T04:09:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc472e1740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc472e1740 1 -- 172.21.15.19:0/2550438518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626cdc53860 con 0x5626cdc44ff0 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.611+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2550438518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc300035f0 con 0x5626cdc44ff0 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2550438518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcc340163d0 con 0x5626cdc44ff0 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc472e1740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc472e1740 10 client.15777 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc472e1740 20 client.15777 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5626cdc54d60 0x5626cdc57150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc472e1740 4 client.15777 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc472e1740 1 -- 172.21.15.19:0/2550438518 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626cdba7770 con 0x5626cdc54d60 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5626cdc54d60 0x5626cdc57150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5626cdc54d60 0x5626cdc57150 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8ac161cd11799e server_cookie=62778fc7a527674 in_seq=0 out_seq=0 2024-09-19T04:09:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.612+0000 7fcc25ffb640 10 client.15777 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2550438518 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626cdba7770 con 0x5626cdc54d60 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc25ffb640 10 client.15777 handle_command_reply: tid=0 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 client.15777 shutdown 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 -- 172.21.15.19:0/2550438518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 msgr2=0x5626cdc42150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc42150 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009c40 tx=0x7fcc34008040 comp rx=0 tx=0).stop 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 -- 172.21.15.19:0/2550438518 shutdown_connections 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc44ff0 0x5626cdc42150 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc36200 0x5626cdc41bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5626cdc54d60 0x5626cdc57150 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2550438518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc44940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc472e1740 1 -- 172.21.15.19:0/2550438518 >> 172.21.15.19:0/2550438518 conn(0x5626cdc31f90 msgr2=0x5626cdc311d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.613+0000 7fcc25ffb640 0 client.15777 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.614+0000 7fcc472e1740 1 -- 172.21.15.19:0/2550438518 shutdown_connections 2024-09-19T04:09:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.614+0000 7fcc472e1740 1 -- 172.21.15.19:0/2550438518 wait complete. 2024-09-19T04:09:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.614+0000 7fcc472e1740 20 client.15777 trim_cache size 0 max 16384 2024-09-19T04:09:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.630+0000 7fcc472e1740 1 Processor -- start 2024-09-19T04:09:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.630+0000 7fcc472e1740 1 -- start start 2024-09-19T04:09:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc31f90 0x5626cdc50e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc5ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc36200 0x5626cdc5cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc472e1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401aca60 con 0x5626cdc36200 2024-09-19T04:09:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc472e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401ae5a0 con 0x5626cdc31f90 2024-09-19T04:09:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc472e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401ad130 con 0x5626cdc358e0 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc36200 0x5626cdc5cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc36200 0x5626cdc5cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45874/0 (socket says 172.21.15.19:45874) 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/1761697899 learned_addr learned my addr 172.21.15.19:0/1761697899 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc5ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc31f90 0x5626cdc50e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3e59c640 1 -- 172.21.15.19:0/1761697899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc31f90 msgr2=0x5626cdc50e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc31f90 0x5626cdc50e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3e59c640 1 -- 172.21.15.19:0/1761697899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc36200 msgr2=0x5626cdc5cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc36200 0x5626cdc5cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3e59c640 1 -- 172.21.15.19:0/1761697899 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626cdc36030 con 0x5626cdc358e0 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc36200 0x5626cdc5cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.631+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc5ca00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5626cdc43230 tx=0x7fcc30031510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cb49775f5b6f245 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc27fff640 1 -- 172.21.15.19:0/1761697899 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30050030 con 0x5626cdc358e0 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc27fff640 1 -- 172.21.15.19:0/1761697899 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc30005d10 con 0x5626cdc358e0 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc27fff640 1 -- 172.21.15.19:0/1761697899 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30035040 con 0x5626cdc358e0 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 -- 172.21.15.19:0/1761697899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 msgr2=0x5626cdc5ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc5ca00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5626cdc43230 tx=0x7fcc30031510 comp rx=0 tx=0).stop 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 -- 172.21.15.19:0/1761697899 shutdown_connections 2024-09-19T04:09:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc36200 0x5626cdc5cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc358e0 0x5626cdc5ca00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 --2- 172.21.15.19:0/1761697899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc31f90 0x5626cdc50e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 -- 172.21.15.19:0/1761697899 >> 172.21.15.19:0/1761697899 conn(0x5626cdc2d520 msgr2=0x5626cdc3d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 -- 172.21.15.19:0/1761697899 shutdown_connections 2024-09-19T04:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.632+0000 7fcc472e1740 1 -- 172.21.15.19:0/1761697899 wait complete. 2024-09-19T04:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 1 -- start start 2024-09-19T04:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 1 Processor -- start 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc31f90 0x5626cdc494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 0x5626cdcbe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401aded0 con 0x5626cdc31f90 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401ad800 con 0x5626cdc358e0 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401aaa40 con 0x5626cdc36200 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc472e1740 10 client.0 _ll_register_callbacks cb 0x7fcc34009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc31f90 0x5626cdc494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc31f90 0x5626cdc494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45888/0 (socket says 172.21.15.19:45888) 2024-09-19T04:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2732779374 learned_addr learned my addr 172.21.15.19:0/2732779374 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2732779374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 msgr2=0x5626cdcbe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 0x5626cdcbe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 msgr2=0x5626cdc499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3dd9b640 1 -- 172.21.15.19:0/2732779374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626cdcbef90 con 0x5626cdc31f90 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.633+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc3dd9b640 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc31f90 0x5626cdc494b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009c40 tx=0x7fcc340098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a583178e6cf571a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-19T04:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2732779374 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34013ea0 con 0x5626cdc31f90 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc472e1740 1 -- 172.21.15.19:0/2732779374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc30005d10 con 0x5626cdc31f90 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2732779374 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fcc340169c0 con 0x5626cdc31f90 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2732779374 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc3403e050 con 0x5626cdc31f90 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc472e1740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc472e1740 1 -- 172.21.15.19:0/2732779374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626cdcc0230 con 0x5626cdc31f90 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2732779374 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc30005d10 con 0x5626cdc31f90 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2732779374 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcc3402fcb0 con 0x5626cdc31f90 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc472e1740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc472e1740 10 client.15783 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc472e1740 20 client.15783 populate_metadata read hostname 'smithi019' 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.634+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5626cdc44ff0 0x5626cdcc28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.635+0000 7fcc472e1740 4 client.15783 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.635+0000 7fcc472e1740 1 -- 172.21.15.19:0/2732779374 --> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5626cdba1c80 con 0x5626cdc44ff0 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.635+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5626cdc44ff0 0x5626cdcc28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.635+0000 7fcc3e59c640 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5626cdc44ff0 0x5626cdcc28b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3421ddb469847a server_cookie=de40f9ea7eaeb1c0 in_seq=0 out_seq=0 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.635+0000 7fcc25ffb640 10 client.15783 ms_handle_connect on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.635+0000 7fcc25ffb640 1 -- 172.21.15.19:0/2732779374 <== mds.0 v2:172.21.15.111:6838/528342006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5626cdba1c80 con 0x5626cdc44ff0 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.635+0000 7fcc25ffb640 10 client.15783 handle_command_reply: tid=0 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.635+0000 7fcc472e1740 1 client.15783 shutdown 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 -- 172.21.15.19:0/2732779374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc31f90 msgr2=0x5626cdc494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc31f90 0x5626cdc494b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009c40 tx=0x7fcc340098d0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 -- 172.21.15.19:0/2732779374 shutdown_connections 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:6838/528342006,v1:172.21.15.111:6839/528342006] conn(0x5626cdc44ff0 0x5626cdcc28b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626cdc36200 0x5626cdcbe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5626cdc358e0 0x5626cdc499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 --2- 172.21.15.19:0/2732779374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626cdc31f90 0x5626cdc494b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 -- 172.21.15.19:0/2732779374 >> 172.21.15.19:0/2732779374 conn(0x5626cdc2d520 msgr2=0x5626cdc309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc25ffb640 0 client.15783 ms_handle_reset on v2:172.21.15.111:6838/528342006 2024-09-19T04:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 -- 172.21.15.19:0/2732779374 shutdown_connections 2024-09-19T04:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 1 -- 172.21.15.19:0/2732779374 wait complete. 2024-09-19T04:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc472e1740 20 client.15783 trim_cache size 0 max 16384 2024-09-19T04:09:36.644 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:09:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 29, 2024-09-19T04:09:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24476, 2024-09-19T04:09:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 12, 2024-09-19T04:09:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": {} 2024-09-19T04:09:36.645 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:09:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 -- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc280a3240 msgr2=0x7fcc280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc280a3240 0x7fcc280a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc3809bcd0 tx=0x7fcc3806ee50 comp rx=0 tx=0).stop 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 -- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc401082a0 msgr2=0x7fcc4019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc401082a0 0x7fcc4019cf20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800cad0 tx=0x7fcc3800ced0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 -- 172.21.15.19:0/4181383305 shutdown_connections 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fcc280a3240 0x7fcc280a5700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc401082a0 0x7fcc4019cf20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc401036e0 0x7fcc40198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 --2- 172.21.15.19:0/4181383305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc40102d30 0x7fcc40198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.636+0000 7fcc44f08640 1 -- 172.21.15.19:0/4181383305 >> 172.21.15.19:0/4181383305 conn(0x7fcc400fc7d0 msgr2=0x7fcc40105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.637+0000 7fcc44f08640 1 -- 172.21.15.19:0/4181383305 shutdown_connections 2024-09-19T04:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:36.637+0000 7fcc44f08640 1 -- 172.21.15.19:0/4181383305 wait complete. 2024-09-19T04:09:36.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:36 smithi111 ceph-mon[35365]: pgmap v440: 129 pgs: 129 active+clean; 229 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s rd, 1.2 MiB/s wr, 471 op/s 2024-09-19T04:09:37.460 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T04:09:37.460 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-19T04:09:37.469 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T04:09:37.478 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T04:09:37.510 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:09:37.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.624+0000 7ffad8b31640 1 -- 172.21.15.19:0/3021599021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 msgr2=0x7ffad4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:37.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.624+0000 7ffad8b31640 1 --2- 172.21.15.19:0/3021599021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 0x7ffad4104a00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.624+0000 7ffad8b31640 1 -- 172.21.15.19:0/3021599021 shutdown_connections 2024-09-19T04:09:37.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.624+0000 7ffad8b31640 1 --2- 172.21.15.19:0/3021599021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4107870 0x7ffad4109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.624+0000 7ffad8b31640 1 --2- 172.21.15.19:0/3021599021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad4104f40 0x7ffad4107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.624+0000 7ffad8b31640 1 --2- 172.21.15.19:0/3021599021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 0x7ffad4104a00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.624+0000 7ffad8b31640 1 -- 172.21.15.19:0/3021599021 >> 172.21.15.19:0/3021599021 conn(0x7ffad40fc1b0 msgr2=0x7ffad40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.625+0000 7ffad8b31640 1 -- 172.21.15.19:0/3021599021 shutdown_connections 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.625+0000 7ffad8b31640 1 -- 172.21.15.19:0/3021599021 wait complete. 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.625+0000 7ffad8b31640 1 Processor -- start 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.625+0000 7ffad8b31640 1 -- start start 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad8b31640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 0x7ffad4101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad8b31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad4104f40 0x7ffad41005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad8b31640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4107870 0x7ffad4100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad8b31640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410d820 con 0x7ffad4102610 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad8b31640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410d6a0 con 0x7ffad4107870 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad8b31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410d9a0 con 0x7ffad4104f40 2024-09-19T04:09:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 0x7ffad4101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 0x7ffad4101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45910/0 (socket says 172.21.15.19:45910) 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 -- 172.21.15.19:0/1588360743 learned_addr learned my addr 172.21.15.19:0/1588360743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffacbfff640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad4104f40 0x7ffad41005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad2d9d640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4107870 0x7ffad4100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 -- 172.21.15.19:0/1588360743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad4104f40 msgr2=0x7ffad41005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad4104f40 0x7ffad41005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 -- 172.21.15.19:0/1588360743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4107870 msgr2=0x7ffad4100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4107870 0x7ffad4100b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 -- 172.21.15.19:0/1588360743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad4101390 con 0x7ffad4102610 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffacbfff640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad4104f40 0x7ffad41005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad259c640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 0x7ffad4101f80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009230 tx=0x7ffac4031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffacb7fe640 1 -- 172.21.15.19:0/1588360743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac403f030 con 0x7ffad4102610 2024-09-19T04:09:37.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffacb7fe640 1 -- 172.21.15.19:0/1588360743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac4003ad0 con 0x7ffad4102610 2024-09-19T04:09:37.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad41a1f10 con 0x7ffad4102610 2024-09-19T04:09:37.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffacb7fe640 1 -- 172.21.15.19:0/1588360743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4035070 con 0x7ffad4102610 2024-09-19T04:09:37.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.626+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41a2370 con 0x7ffad4102610 2024-09-19T04:09:37.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.627+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad40fdca0 con 0x7ffad4102610 2024-09-19T04:09:37.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.630+0000 7ffacb7fe640 1 -- 172.21.15.19:0/1588360743 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ffac4042030 con 0x7ffad4102610 2024-09-19T04:09:37.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.630+0000 7ffacb7fe640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a32d0 0x7ffac00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:37.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.630+0000 7ffacbfff640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a32d0 0x7ffac00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:37.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.630+0000 7ffacb7fe640 1 -- 172.21.15.19:0/1588360743 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7ffac40f05d0 con 0x7ffad4102610 2024-09-19T04:09:37.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.631+0000 7ffacbfff640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a32d0 0x7ffac00a5790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffad4101cf0 tx=0x7ffabc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:37.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.633+0000 7ffacb7fe640 1 -- 172.21.15.19:0/1588360743 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffac40bcf50 con 0x7ffad4102610 2024-09-19T04:09:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.809+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffad402da10 con 0x7ffad4102610 2024-09-19T04:09:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.810+0000 7ffacb7fe640 1 -- 172.21.15.19:0/1588360743 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24842 (secure 0 0 0) 0x7ffac40bc850 con 0x7ffad4102610 2024-09-19T04:09:37.812 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:09:37.812 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":93,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:06:42.902449+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:09:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a32d0 msgr2=0x7ffac00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a32d0 0x7ffac00a5790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffad4101cf0 tx=0x7ffabc005f90 comp rx=0 tx=0).stop 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 msgr2=0x7ffad4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 0x7ffad4101f80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009230 tx=0x7ffac4031b60 comp rx=0 tx=0).stop 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 shutdown_connections 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7ffac00a32d0 0x7ffac00a5790 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad4107870 0x7ffad4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffad4104f40 0x7ffad41005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 --2- 172.21.15.19:0/1588360743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad4102610 0x7ffad4101f80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 >> 172.21.15.19:0/1588360743 conn(0x7ffad40fc1b0 msgr2=0x7ffad4102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 shutdown_connections 2024-09-19T04:09:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.812+0000 7ffad8b31640 1 -- 172.21.15.19:0/1588360743 wait complete. 2024-09-19T04:09:37.825 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T03:59:58.845133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T04:04:36.009364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T04:04:38.289030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-19T04:04:44.768678+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-19T04:09:37.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:09:37.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 -- 172.21.15.19:0/862403008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588100a30 msgr2=0x7f8588100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:37.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 --2- 172.21.15.19:0/862403008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588100a30 0x7f8588100e30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8578009900 tx=0x7f857802a410 comp rx=0 tx=0).stop 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 -- 172.21.15.19:0/862403008 shutdown_connections 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 --2- 172.21.15.19:0/862403008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85881025e0 0x7f8588108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 --2- 172.21.15.19:0/862403008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8588101c20 0x7f85881020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 --2- 172.21.15.19:0/862403008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588100a30 0x7f8588100e30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 -- 172.21.15.19:0/862403008 >> 172.21.15.19:0/862403008 conn(0x7f85880fc1b0 msgr2=0x7f85880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 -- 172.21.15.19:0/862403008 shutdown_connections 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.932+0000 7f858e29b640 1 -- 172.21.15.19:0/862403008 wait complete. 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858e29b640 1 Processor -- start 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858e29b640 1 -- start start 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858e29b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588100a30 0x7f858806aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858e29b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588101c20 0x7f858819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858e29b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85881025e0 0x7f858819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858e29b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f858810c6e0 con 0x7f8588101c20 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858e29b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f858810c560 con 0x7f8588100a30 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858e29b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f858810c860 con 0x7f85881025e0 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f85877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588101c20 0x7f858819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f85877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588101c20 0x7f858819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45926/0 (socket says 172.21.15.19:45926) 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f85877fe640 1 -- 172.21.15.19:0/404100354 learned_addr learned my addr 172.21.15.19:0/404100354 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f8587fff640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588100a30 0x7f858806aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f858c838640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85881025e0 0x7f858819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f85877fe640 1 -- 172.21.15.19:0/404100354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85881025e0 msgr2=0x7f858819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f85877fe640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85881025e0 0x7f858819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f85877fe640 1 -- 172.21.15.19:0/404100354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588100a30 msgr2=0x7f858806aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f85877fe640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588100a30 0x7f858806aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.933+0000 7f85877fe640 1 -- 172.21.15.19:0/404100354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858819fba0 con 0x7f8588101c20 2024-09-19T04:09:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.934+0000 7f85877fe640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588101c20 0x7f858819cc70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f857c00d950 tx=0x7f857c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.934+0000 7f85857fa640 1 -- 172.21.15.19:0/404100354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c017070 con 0x7f8588101c20 2024-09-19T04:09:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.934+0000 7f85857fa640 1 -- 172.21.15.19:0/404100354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c009c30 con 0x7f8588101c20 2024-09-19T04:09:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.934+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858819fe30 con 0x7f8588101c20 2024-09-19T04:09:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.934+0000 7f85857fa640 1 -- 172.21.15.19:0/404100354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c00f6b0 con 0x7f8588101c20 2024-09-19T04:09:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.934+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881a02f0 con 0x7f8588101c20 2024-09-19T04:09:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.935+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8588107eb0 con 0x7f8588101c20 2024-09-19T04:09:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.936+0000 7f85857fa640 1 -- 172.21.15.19:0/404100354 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f857c0109d0 con 0x7f8588101c20 2024-09-19T04:09:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.937+0000 7f85857fa640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85700a3250 0x7f85700a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.937+0000 7f85857fa640 1 -- 172.21.15.19:0/404100354 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f857c0cf830 con 0x7f8588101c20 2024-09-19T04:09:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.937+0000 7f8587fff640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85700a3250 0x7f85700a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.937+0000 7f8587fff640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85700a3250 0x7f85700a5710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8578005ab0 tx=0x7f8578005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:37.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:37.941+0000 7f85857fa640 1 -- 172.21.15.19:0/404100354 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f857c009de0 con 0x7f8588101c20 2024-09-19T04:09:38.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.119+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8588062e90 con 0x7f8588101c20 2024-09-19T04:09:38.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.119+0000 7f85857fa640 1 -- 172.21.15.19:0/404100354 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f857c09bb60 con 0x7f8588101c20 2024-09-19T04:09:38.120 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85700a3250 msgr2=0x7f85700a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85700a3250 0x7f85700a5710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8578005ab0 tx=0x7f8578005a20 comp rx=0 tx=0).stop 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588101c20 msgr2=0x7f858819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588101c20 0x7f858819cc70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f857c00d950 tx=0x7f857c00de10 comp rx=0 tx=0).stop 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 shutdown_connections 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f85700a3250 0x7f85700a5710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85881025e0 0x7f858819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8588101c20 0x7f858819cc70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 --2- 172.21.15.19:0/404100354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588100a30 0x7f858806aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 >> 172.21.15.19:0/404100354 conn(0x7f85880fc1b0 msgr2=0x7f8588105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 shutdown_connections 2024-09-19T04:09:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.121+0000 7f858e29b640 1 -- 172.21.15.19:0/404100354 wait complete. 2024-09-19T04:09:38.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T04:09:38.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 -- 172.21.15.19:0/3945277583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 msgr2=0x7f89fc101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3945277583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 0x7f89fc101f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009900 tx=0x7f89ec02a410 comp rx=0 tx=0).stop 2024-09-19T04:09:38.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 -- 172.21.15.19:0/3945277583 shutdown_connections 2024-09-19T04:09:38.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3945277583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1036d0 0x7f89fc109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3945277583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89fc102d10 0x7f89fc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3945277583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 0x7f89fc101f20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 -- 172.21.15.19:0/3945277583 >> 172.21.15.19:0/3945277583 conn(0x7f89fc0fd2a0 msgr2=0x7f89fc0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:38.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 -- 172.21.15.19:0/3945277583 shutdown_connections 2024-09-19T04:09:38.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.241+0000 7f8a0095f640 1 -- 172.21.15.19:0/3945277583 wait complete. 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f8a0095f640 1 Processor -- start 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f8a0095f640 1 -- start start 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f8a0095f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 0x7f89fc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f8a0095f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89fc102d10 0x7f89fc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f8a0095f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1036d0 0x7f89fc19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f8a0095f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc10d7f0 con 0x7f89fc101b20 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f8a0095f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc10d670 con 0x7f89fc102d10 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f8a0095f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc10d970 con 0x7f89fc1036d0 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f89fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 0x7f89fc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f89fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 0x7f89fc197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45942/0 (socket says 172.21.15.19:45942) 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f89fa59c640 1 -- 172.21.15.19:0/3269472422 learned_addr learned my addr 172.21.15.19:0/3269472422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f89f3fff640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89fc102d10 0x7f89fc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.242+0000 7f89fad9d640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1036d0 0x7f89fc19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89fa59c640 1 -- 172.21.15.19:0/3269472422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1036d0 msgr2=0x7f89fc19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89fa59c640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1036d0 0x7f89fc19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89fa59c640 1 -- 172.21.15.19:0/3269472422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89fc102d10 msgr2=0x7f89fc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89fa59c640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89fc102d10 0x7f89fc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89fa59c640 1 -- 172.21.15.19:0/3269472422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc19fac0 con 0x7f89fc101b20 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89fad9d640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1036d0 0x7f89fc19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89fa59c640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 0x7f89fc197ea0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f89ec031440 tx=0x7f89ec032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89f37fe640 1 -- 172.21.15.19:0/3269472422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec03f030 con 0x7f89fc101b20 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89f37fe640 1 -- 172.21.15.19:0/3269472422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ec003ce0 con 0x7f89fc101b20 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc19fd50 con 0x7f89fc101b20 2024-09-19T04:09:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f89f37fe640 1 -- 172.21.15.19:0/3269472422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec035070 con 0x7f89fc101b20 2024-09-19T04:09:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.243+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc1a01d0 con 0x7f89fc101b20 2024-09-19T04:09:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.244+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89fc108f10 con 0x7f89fc101b20 2024-09-19T04:09:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.245+0000 7f89f37fe640 1 -- 172.21.15.19:0/3269472422 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f89ec042030 con 0x7f89fc101b20 2024-09-19T04:09:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.246+0000 7f89f37fe640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89e80a32d0 0x7f89e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.246+0000 7f89f3fff640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89e80a32d0 0x7f89e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.246+0000 7f89f37fe640 1 -- 172.21.15.19:0/3269472422 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f89ec0f0670 con 0x7f89fc101b20 2024-09-19T04:09:38.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.247+0000 7f89f3fff640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89e80a32d0 0x7f89e80a5790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89fc1993b0 tx=0x7f89e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:38.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.249+0000 7f89f37fe640 1 -- 172.21.15.19:0/3269472422 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f89ec0bcf90 con 0x7f89fc101b20 2024-09-19T04:09:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.424+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89fc062e90 con 0x7f89fc101b20 2024-09-19T04:09:38.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.425+0000 7f89f37fe640 1 -- 172.21.15.19:0/3269472422 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f89ec0bc890 con 0x7f89fc101b20 2024-09-19T04:09:38.429 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89e80a32d0 msgr2=0x7f89e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89e80a32d0 0x7f89e80a5790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89fc1993b0 tx=0x7f89e400b390 comp rx=0 tx=0).stop 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 msgr2=0x7f89fc197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 0x7f89fc197ea0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f89ec031440 tx=0x7f89ec032e20 comp rx=0 tx=0).stop 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 shutdown_connections 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f89e80a32d0 0x7f89e80a5790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1036d0 0x7f89fc19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89fc102d10 0x7f89fc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 --2- 172.21.15.19:0/3269472422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101b20 0x7f89fc197ea0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 >> 172.21.15.19:0/3269472422 conn(0x7f89fc0fd2a0 msgr2=0x7f89fc1060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 shutdown_connections 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.427+0000 7f8a0095f640 1 -- 172.21.15.19:0/3269472422 wait complete. 2024-09-19T04:09:38.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:38 smithi019 ceph-mon[30938]: pgmap v441: 129 pgs: 129 active+clean; 157 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-09-19T04:09:38.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:38 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1588360743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:09:38.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:38 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/404100354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:09:38.439 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T04:09:38.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:38 smithi196 ceph-mon[35731]: pgmap v441: 129 pgs: 129 active+clean; 157 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-09-19T04:09:38.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:38 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1588360743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:09:38.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:38 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/404100354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2697844683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e58101c20 msgr2=0x7f7e581020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2697844683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e58101c20 0x7f7e581020a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009900 tx=0x7f7e4802a410 comp rx=0 tx=0).stop 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2697844683 shutdown_connections 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2697844683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e581025e0 0x7f7e58108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2697844683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e58101c20 0x7f7e581020a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2697844683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100a30 0x7f7e58100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2697844683 >> 172.21.15.19:0/2697844683 conn(0x7f7e580fc1b0 msgr2=0x7f7e580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2697844683 shutdown_connections 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.543+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2697844683 wait complete. 2024-09-19T04:09:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5fc77640 1 Processor -- start 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5fc77640 1 -- start start 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5fc77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100a30 0x7f7e581049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5fc77640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e58101c20 0x7f7e58103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5fc77640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e581025e0 0x7f7e58103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5fc77640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810c6e0 con 0x7f7e581025e0 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5fc77640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810c560 con 0x7f7e58101c20 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5fc77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810c860 con 0x7f7e58100a30 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5e214640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e581025e0 0x7f7e58103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5e214640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e581025e0 0x7f7e58103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45960/0 (socket says 172.21.15.19:45960) 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5e214640 1 -- 172.21.15.19:0/2999477067 learned_addr learned my addr 172.21.15.19:0/2999477067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5da13640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100a30 0x7f7e581049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.544+0000 7f7e5d212640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e58101c20 0x7f7e58103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e5e214640 1 -- 172.21.15.19:0/2999477067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100a30 msgr2=0x7f7e581049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e5e214640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100a30 0x7f7e581049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e5e214640 1 -- 172.21.15.19:0/2999477067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e58101c20 msgr2=0x7f7e58103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e5e214640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e58101c20 0x7f7e58103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e5e214640 1 -- 172.21.15.19:0/2999477067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e58103dd0 con 0x7f7e581025e0 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e5e214640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e581025e0 0x7f7e58103540 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000eaa0 tx=0x7f7e5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e4effd640 1 -- 172.21.15.19:0/2999477067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000cdb0 con 0x7f7e581025e0 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e4effd640 1 -- 172.21.15.19:0/2999477067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e50005020 con 0x7f7e581025e0 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e4effd640 1 -- 172.21.15.19:0/2999477067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500116b0 con 0x7f7e581025e0 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e581a1fe0 con 0x7f7e581025e0 2024-09-19T04:09:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.545+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e581a2450 con 0x7f7e581025e0 2024-09-19T04:09:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.546+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e581a2740 con 0x7f7e581025e0 2024-09-19T04:09:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.547+0000 7f7e4effd640 1 -- 172.21.15.19:0/2999477067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7e50005560 con 0x7f7e581025e0 2024-09-19T04:09:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.548+0000 7f7e4effd640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7e440a32d0 0x7f7e440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.548+0000 7f7e5da13640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7e440a32d0 0x7f7e440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.548+0000 7f7e4effd640 1 -- 172.21.15.19:0/2999477067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f7e500cfe00 con 0x7f7e581025e0 2024-09-19T04:09:38.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.548+0000 7f7e5da13640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7e440a32d0 0x7f7e440a5790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7e58101a80 tx=0x7f7e5400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:38.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.552+0000 7f7e4effd640 1 -- 172.21.15.19:0/2999477067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7e50016030 con 0x7f7e581025e0 2024-09-19T04:09:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:38 smithi111 ceph-mon[35365]: pgmap v441: 129 pgs: 129 active+clean; 157 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-09-19T04:09:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:38 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1588360743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:09:38.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:38 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/404100354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T04:09:38.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.728+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7e581a2910 con 0x7f7e581025e0 2024-09-19T04:09:38.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.728+0000 7f7e4effd640 1 -- 172.21.15.19:0/2999477067 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f7e5009c130 con 0x7f7e581025e0 2024-09-19T04:09:38.730 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-09-19T04:09:38.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.730+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7e440a32d0 msgr2=0x7f7e440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.730+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7e440a32d0 0x7f7e440a5790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7e58101a80 tx=0x7f7e5400b390 comp rx=0 tx=0).stop 2024-09-19T04:09:38.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e581025e0 msgr2=0x7f7e58103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e581025e0 0x7f7e58103540 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000eaa0 tx=0x7f7e5000ef60 comp rx=0 tx=0).stop 2024-09-19T04:09:38.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 shutdown_connections 2024-09-19T04:09:38.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7e440a32d0 0x7f7e440a5790 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e581025e0 0x7f7e58103540 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e58101c20 0x7f7e58103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 --2- 172.21.15.19:0/2999477067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100a30 0x7f7e581049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 >> 172.21.15.19:0/2999477067 conn(0x7f7e580fc1b0 msgr2=0x7f7e5806a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:38.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 shutdown_connections 2024-09-19T04:09:38.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.731+0000 7f7e5fc77640 1 -- 172.21.15.19:0/2999477067 wait complete. 2024-09-19T04:09:38.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-19T04:09:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.848+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3388798821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 msgr2=0x7f7b381072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.848+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3388798821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 0x7f7b381072d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009900 tx=0x7f7b2002a410 comp rx=0 tx=0).stop 2024-09-19T04:09:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.848+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3388798821 shutdown_connections 2024-09-19T04:09:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.848+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3388798821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b38107810 0x7f7b38109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.848+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3388798821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 0x7f7b381072d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.848+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3388798821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b381025b0 0x7f7b381049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.848+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3388798821 >> 172.21.15.19:0/3388798821 conn(0x7f7b380fc1b0 msgr2=0x7f7b380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3388798821 shutdown_connections 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3388798821 wait complete. 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 Processor -- start 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 -- start start 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b381025b0 0x7f7b38101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 0x7f7b38100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b38107810 0x7f7b38100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3810d7c0 con 0x7f7b38104ee0 2024-09-19T04:09:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3810d640 con 0x7f7b381025b0 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.849+0000 7f7b3de8b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3810d940 con 0x7f7b38107810 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 0x7f7b38100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 0x7f7b38100560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45970/0 (socket says 172.21.15.19:45970) 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 -- 172.21.15.19:0/3568632290 learned_addr learned my addr 172.21.15.19:0/3568632290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b37fff640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b38107810 0x7f7b38100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 -- 172.21.15.19:0/3568632290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b38107810 msgr2=0x7f7b38100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b377fe640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b381025b0 0x7f7b38101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b38107810 0x7f7b38100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 -- 172.21.15.19:0/3568632290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b381025b0 msgr2=0x7f7b38101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b381025b0 0x7f7b38101f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 -- 172.21.15.19:0/3568632290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b38101330 con 0x7f7b38104ee0 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b37fff640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b38107810 0x7f7b38100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:09:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b2effd640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 0x7f7b38100560 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7b20006010 tx=0x7f7b20031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b357fa640 1 -- 172.21.15.19:0/3568632290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2003f030 con 0x7f7b38104ee0 2024-09-19T04:09:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b357fa640 1 -- 172.21.15.19:0/3568632290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b200044a0 con 0x7f7b38104ee0 2024-09-19T04:09:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b381a1f90 con 0x7f7b38104ee0 2024-09-19T04:09:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b357fa640 1 -- 172.21.15.19:0/3568632290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20035070 con 0x7f7b38104ee0 2024-09-19T04:09:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.850+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b381a23f0 con 0x7f7b38104ee0 2024-09-19T04:09:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.851+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b380fdca0 con 0x7f7b38104ee0 2024-09-19T04:09:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.852+0000 7f7b357fa640 1 -- 172.21.15.19:0/3568632290 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7b20042030 con 0x7f7b38104ee0 2024-09-19T04:09:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.853+0000 7f7b357fa640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7b240a3250 0x7f7b240a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.853+0000 7f7b377fe640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7b240a3250 0x7f7b240a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.853+0000 7f7b357fa640 1 -- 172.21.15.19:0/3568632290 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f7b200f04a0 con 0x7f7b38104ee0 2024-09-19T04:09:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.854+0000 7f7b377fe640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7b240a3250 0x7f7b240a5710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7b28005fd0 tx=0x7f7b28005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:38.856+0000 7f7b357fa640 1 -- 172.21.15.19:0/3568632290 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b200bcdc0 con 0x7f7b38104ee0 2024-09-19T04:09:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.034+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7b3802da10 con 0x7f7b38104ee0 2024-09-19T04:09:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.035+0000 7f7b357fa640 1 -- 172.21.15.19:0/3568632290 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f7b200bc6c0 con 0x7f7b38104ee0 2024-09-19T04:09:39.036 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-09-19T04:09:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7b240a3250 msgr2=0x7f7b240a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7b240a3250 0x7f7b240a5710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7b28005fd0 tx=0x7f7b28005c70 comp rx=0 tx=0).stop 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 msgr2=0x7f7b38100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 0x7f7b38100560 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7b20006010 tx=0x7f7b20031b60 comp rx=0 tx=0).stop 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 shutdown_connections 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f7b240a3250 0x7f7b240a5710 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b38107810 0x7f7b38100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b38104ee0 0x7f7b38100560 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 --2- 172.21.15.19:0/3568632290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b381025b0 0x7f7b38101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 >> 172.21.15.19:0/3568632290 conn(0x7f7b380fc1b0 msgr2=0x7f7b38102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 shutdown_connections 2024-09-19T04:09:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.037+0000 7f7b3de8b640 1 -- 172.21.15.19:0/3568632290 wait complete. 2024-09-19T04:09:39.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.151+0000 7f1229a53640 1 -- 172.21.15.19:0/3975420805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224102620 msgr2=0x7f1224108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.151+0000 7f1229a53640 1 --2- 172.21.15.19:0/3975420805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224102620 0x7f1224108eb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f121c009ca0 tx=0x7f121c02a950 comp rx=0 tx=0).stop 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 -- 172.21.15.19:0/3975420805 shutdown_connections 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 --2- 172.21.15.19:0/3975420805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224102620 0x7f1224108eb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 --2- 172.21.15.19:0/3975420805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1224101c60 0x7f12241020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 --2- 172.21.15.19:0/3975420805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224100f50 0x7f1224101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 -- 172.21.15.19:0/3975420805 >> 172.21.15.19:0/3975420805 conn(0x7f12240fb950 msgr2=0x7f12240fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 -- 172.21.15.19:0/3975420805 shutdown_connections 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 -- 172.21.15.19:0/3975420805 wait complete. 2024-09-19T04:09:39.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 Processor -- start 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 -- start start 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224100f50 0x7f1224197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224101c60 0x7f1224198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1224102620 0x7f122419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f122410c6d0 con 0x7f1224101c60 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f122410c550 con 0x7f1224102620 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.152+0000 7f1229a53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f122410c850 con 0x7f1224100f50 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224101c60 0x7f1224198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224101c60 0x7f1224198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45978/0 (socket says 172.21.15.19:45978) 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 -- 172.21.15.19:0/1956797441 learned_addr learned my addr 172.21.15.19:0/1956797441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f1222ffd640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224100f50 0x7f1224197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:39.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f12237fe640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1224102620 0x7f122419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 -- 172.21.15.19:0/1956797441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224100f50 msgr2=0x7f1224197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224100f50 0x7f1224197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 -- 172.21.15.19:0/1956797441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1224102620 msgr2=0x7f122419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1224102620 0x7f122419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 -- 172.21.15.19:0/1956797441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122419fa50 con 0x7f1224101c60 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f121bfff640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224101c60 0x7f1224198370 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f120c00e9f0 tx=0x7f120c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f1228a51640 1 -- 172.21.15.19:0/1956797441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c00cd80 con 0x7f1224101c60 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f1228a51640 1 -- 172.21.15.19:0/1956797441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c010910 con 0x7f1224101c60 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122419fd40 con 0x7f1224101c60 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f1228a51640 1 -- 172.21.15.19:0/1956797441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c018990 con 0x7f1224101c60 2024-09-19T04:09:39.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.153+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12241a0280 con 0x7f1224101c60 2024-09-19T04:09:39.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.154+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12241a04b0 con 0x7f1224101c60 2024-09-19T04:09:39.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.156+0000 7f1228a51640 1 -- 172.21.15.19:0/1956797441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f120c010ac0 con 0x7f1224101c60 2024-09-19T04:09:39.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.156+0000 7f1228a51640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f12100a32d0 0x7f12100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:09:39.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.157+0000 7f1222ffd640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f12100a32d0 0x7f12100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:09:39.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.157+0000 7f1228a51640 1 -- 172.21.15.19:0/1956797441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f120c0cfff0 con 0x7f1224101c60 2024-09-19T04:09:39.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.157+0000 7f1222ffd640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f12100a32d0 0x7f12100a5790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1214005fd0 tx=0x7f121400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:09:39.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.161+0000 7f1228a51640 1 -- 172.21.15.19:0/1956797441 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f120c09c950 con 0x7f1224101c60 2024-09-19T04:09:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.388+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f12241a0da0 con 0x7f1224101c60 2024-09-19T04:09:39.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.395+0000 7f1228a51640 1 -- 172.21.15.19:0/1956797441 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+742341 (secure 0 0 0) 0x7f120c09c250 con 0x7f1224101c60 2024-09-19T04:09:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:09:39.407 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:09:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 20, 2024-09-19T04:09:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 14150, 2024-09-19T04:09:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2387193681 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2387193681 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/2387193681", 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-09-19T03:57:40.326691+0000", 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24112, 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-09-19T04:09:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-09-19T04:09:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-09-19T04:09:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:09:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-09-19T04:09:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-09-19T04:09:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:09:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-09-19T04:09:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-09-19T04:09:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-09-19T04:09:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:09:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-09-19T04:09:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:09:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:09:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:09:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:09:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-09-19T04:09:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-09-19T04:09:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:09:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:09:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:09:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:09:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-09-19T04:09:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:09:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-09-19T04:09:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:09:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-09-19T04:09:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-09-19T04:09:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-09-19T04:09:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:09:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:09:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:09:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:09:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:09:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:09:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:09:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:09:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:09:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-09-19T04:09:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:09:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-09-19T04:09:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-09-19T04:09:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-09-19T04:09:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-09-19T04:09:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-09-19T04:09:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:09:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-09-19T04:09:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-19T04:09:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-09-19T04:09:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-09-19T04:09:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:09:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:09:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-09-19T04:09:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-09-19T04:09:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-09-19T04:09:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-09-19T04:09:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:09:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:09:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-09-19T04:09:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-09-19T04:09:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-09-19T04:09:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24116, 2024-09-19T04:09:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-09-19T04:09:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-09-19T04:09:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:09:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-09-19T04:09:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:09:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-09-19T04:09:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-09-19T04:09:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-09-19T04:09:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:09:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:09:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-09-19T04:09:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-09-19T04:09:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-09-19T04:09:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:09:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-09-19T04:09:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:09:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:09:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-09-19T04:09:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:09:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:09:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:09:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-19T04:09:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-09-19T04:09:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-09-19T04:09:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-09-19T04:09:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-09-19T04:09:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:09:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-09-19T04:09:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-09-19T04:09:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-09-19T04:09:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:09:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-19T04:09:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-09-19T04:09:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-09-19T04:09:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-09-19T04:09:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-09-19T04:09:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:09:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:09:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-09-19T04:09:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "restful" 2024-09-19T04:09:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-09-19T04:09:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-09-19T04:09:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-09-19T04:09:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T04:09:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-19T04:09:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-09-19T04:09:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-09-19T04:09:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-09-19T04:09:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T04:09:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T04:09:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-09-19T04:09:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T04:09:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T04:09:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-09-19T04:09:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-09-19T04:09:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-09-19T04:09:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T04:09:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-09-19T04:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T04:09:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-09-19T04:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-09-19T04:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-09-19T04:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-09-19T04:09:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-09-19T04:09:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-09-19T04:09:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T04:09:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-09-19T04:09:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-09-19T04:09:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.451 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-09-19T04:09:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-09-19T04:09:40.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.465 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.466 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T04:09:40.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T04:09:40.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T04:09:40.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.484 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T04:09:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T04:09:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T04:09:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T04:09:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T04:09:40.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T04:09:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-09-19T04:09:40.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T04:09:40.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T04:09:40.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.515 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.525 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-19T04:09:40.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-09-19T04:09:40.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-19T04:09:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.542 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.544 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.545 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-09-19T04:09:40.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.550 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.559 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-09-19T04:09:40.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-09-19T04:09:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-09-19T04:09:40.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.570 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.572 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-19T04:09:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-09-19T04:09:40.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-09-19T04:09:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.585 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.586 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.587 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.591 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.593 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-09-19T04:09:40.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-09-19T04:09:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.603 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-09-19T04:09:40.612 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T04:09:40.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.622 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.628 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.633 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.635 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.654 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.662 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.665 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.668 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-19T04:09:40.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.672 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.676 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-19T04:09:40.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-19T04:09:40.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.691 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.699 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.702 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.704 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.705 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-19T04:09:40.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.709 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.713 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.715 INFO:teuthology.orchestra.run.smithi019.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.725 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.729 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-19T04:09:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-09-19T04:09:40.735 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-09-19T04:09:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.746 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.750 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-09-19T04:09:40.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.774 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T04:09:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.789 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.791 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-09-19T04:09:40.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.798 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-09-19T04:09:40.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.810 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-09-19T04:09:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.814 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.815 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-19T04:09:40.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-19T04:09:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-19T04:09:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-19T04:09:40.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-19T04:09:40.824 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-19T04:09:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-19T04:09:40.826 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-19T04:09:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-19T04:09:40.828 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 3, 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2224928695 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:40.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2156450853 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-09-19T04:09:40.830 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1548003543 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-19T04:09:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-09-19T04:09:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1839666147 2024-09-19T04:09:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-19T04:09:40.832 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f12100a32d0 msgr2=0x7f12100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f12100a32d0 0x7f12100a5790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1214005fd0 tx=0x7f121400b390 comp rx=0 tx=0).stop 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224101c60 msgr2=0x7f1224198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224101c60 0x7f1224198370 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f120c00e9f0 tx=0x7f120c00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 shutdown_connections 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f12100a32d0 0x7f12100a5790 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1224102620 0x7f122419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224101c60 0x7f1224198370 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 --2- 172.21.15.19:0/1956797441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224100f50 0x7f1224197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 >> 172.21.15.19:0/1956797441 conn(0x7f12240fb950 msgr2=0x7f12240fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:09:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 shutdown_connections 2024-09-19T04:09:40.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:09:39.401+0000 7f1229a53640 1 -- 172.21.15.19:0/1956797441 wait complete. 2024-09-19T04:09:40.835 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:39 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3269472422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.835 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:39 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2999477067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.835 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:39 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3568632290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.835 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:40 smithi111 ceph-mon[35365]: pgmap v442: 129 pgs: 129 active+clean; 89 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 823 KiB/s wr, 367 op/s 2024-09-19T04:09:40.835 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:40 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1956797441' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:09:40.835 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:39 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3269472422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.835 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:39 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2999477067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.835 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:39 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3568632290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.836 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:40 smithi196 ceph-mon[35731]: pgmap v442: 129 pgs: 129 active+clean; 89 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 823 KiB/s wr, 367 op/s 2024-09-19T04:09:40.836 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:40 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1956797441' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:09:40.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:39 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3269472422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:39 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2999477067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:39 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3568632290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T04:09:40.896 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T04:09:40.897 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T04:09:40.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:40 smithi019 ceph-mon[30938]: pgmap v442: 129 pgs: 129 active+clean; 89 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 823 KiB/s wr, 367 op/s 2024-09-19T04:09:40.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:40 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1956797441' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T04:09:41.170 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:41.676 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:09:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 363, 2024-09-19T04:09:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 356, 2024-09-19T04:09:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 838848, 2024-09-19T04:09:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 136730, 2024-09-19T04:09:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-09-19T04:09:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-09-19T04:09:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.041666815, 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.014514429, 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.021613714, 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.007120435, 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 355, 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.015920404, 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000044846 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 751824, 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 54368 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 153, 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 216, 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 40292, 2024-09-19T04:09:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 231100, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.020460288, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.007150717, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.006818873, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004850206, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 215, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.013036381, 2024-09-19T04:09:41.680 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000060634 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 183968 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:41.681 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 263, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 265, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 69795, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 83635, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.020732152, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.006592365, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.006083967, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.006005214, 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:41.682 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 263, 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.009931660, 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000037762 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:09:41.683 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1159, 2024-09-19T04:09:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.029085972, 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000025095 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.685 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-09-19T04:09:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-09-19T04:09:41.687 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-09-19T04:09:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-09-19T04:09:41.689 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:09:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-09-19T04:09:41.691 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-09-19T04:09:41.692 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-09-19T04:09:41.694 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 3, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-09-19T04:09:41.695 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4195200, 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4194892, 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 3 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-09-19T04:09:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-09-19T04:09:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 48484, 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215328 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.698 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-09-19T04:09:41.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.702 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.707 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-09-19T04:09:41.708 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.709 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-09-19T04:09:41.712 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1228, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:41.713 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:41.714 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:41.714 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:41.714 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:41.714 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:41.714 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:41.714 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:41.714 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:41.715 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:41.716 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 189239, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 48, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-19T04:09:41.717 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 168, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8512, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-09-19T04:09:41.718 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:41.719 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-19T04:09:41.719 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:41.719 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-19T04:09:41.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 670, 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 670, 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 670, 2024-09-19T04:09:41.721 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.393996067, 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000588053 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 670, 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 671 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 670, 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 670, 2024-09-19T04:09:41.722 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 404, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 264, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-09-19T04:09:41.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:41.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-09-19T04:09:41.725 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 93, 2024-09-19T04:09:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 4, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 5, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 5, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.727 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-19T04:09:41.729 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 775, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 870816, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 775, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 775, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 870816, 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:41.730 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-19T04:09:41.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 670, 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 901, 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 670, 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 901, 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.732 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:41.733 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 670, 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 670, 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 670, 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 670, 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:41.734 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-19T04:09:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55949753ab40": { 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-19T04:09:41.736 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.738 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:41.738 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:09:42.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:42 smithi111 ceph-mon[35365]: pgmap v443: 129 pgs: 129 active+clean; 4.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 762 KiB/s wr, 331 op/s 2024-09-19T04:09:42.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:42 smithi019 ceph-mon[30938]: pgmap v443: 129 pgs: 129 active+clean; 4.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 762 KiB/s wr, 331 op/s 2024-09-19T04:09:42.690 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T04:09:42.690 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T04:09:42.690 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-19T04:09:42.690 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-19T04:09:42.691 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T04:09:42.691 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T04:09:42.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:42 smithi196 ceph-mon[35731]: pgmap v443: 129 pgs: 129 active+clean; 4.8 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 762 KiB/s wr, 331 op/s 2024-09-19T04:09:42.949 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:43.460 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:09:43.460 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:43.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 102, 2024-09-19T04:09:43.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 95, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 724464, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 56480, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.022010283, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.006450022, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.020054263, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 95, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.009034893, 2024-09-19T04:09:43.461 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000095104 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 724464, 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 56480 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:43.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 71, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 160, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 91088, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 3, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.005284552, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.003520482, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.001101069, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 69, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002918455, 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000042296 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 91088 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.000290947, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000196819, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:43.467 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148863, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 28, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-19T04:09:43.469 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-09-19T04:09:43.470 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 99, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 707573, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 99, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-19T04:09:43.471 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 99, 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 707573, 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:43.472 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:09:44.160 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T04:09:44.160 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T04:09:44.160 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-19T04:09:44.160 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-19T04:09:44.161 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T04:09:44.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T04:09:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:44 smithi019 ceph-mon[30938]: pgmap v444: 129 pgs: 129 active+clean; 4.7 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 422 KiB/s wr, 217 op/s 2024-09-19T04:09:44.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:44 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:44.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:44 smithi111 ceph-mon[35365]: pgmap v444: 129 pgs: 129 active+clean; 4.7 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 422 KiB/s wr, 217 op/s 2024-09-19T04:09:44.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:44 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:44.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:44 smithi196 ceph-mon[35731]: pgmap v444: 129 pgs: 129 active+clean; 4.7 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 422 KiB/s wr, 217 op/s 2024-09-19T04:09:44.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:44 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:09:44.954 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:09:44.954 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 368, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 361, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 840400, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 138964, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.047449481, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.018389071, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.024789098, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.007070211, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 360, 2024-09-19T04:09:44.955 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.017260188, 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000047944 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 752368, 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 55648 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:44.956 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 456, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 521, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 119864, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 329411, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.033813570, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.011544723, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.010222241, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.009574777, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 520, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.019047245, 2024-09-19T04:09:44.957 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000036629 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 185936 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 6, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1629, 2024-09-19T04:09:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 1284, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001418371, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000061145, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.001158606, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000108162, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002203406, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000550851 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:44.959 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-19T04:09:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1241, 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.027909970, 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000022489 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.961 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:44.962 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-09-19T04:09:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-09-19T04:09:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:44.965 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-09-19T04:09:44.966 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-09-19T04:09:44.968 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 5, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-09-19T04:09:44.969 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4195734, 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4195426, 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 5 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-09-19T04:09:44.970 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 48380, 2024-09-19T04:09:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215328 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.973 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.976 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.977 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.977 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-09-19T04:09:44.977 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.977 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-09-19T04:09:44.978 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.979 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.980 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:44.981 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 2, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 1, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-09-19T04:09:44.984 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1235, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:44.985 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:44.986 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:44.987 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 197359, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 53, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 168, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8512, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 717, 2024-09-19T04:09:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 717, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 717, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.339996619, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000474193 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 717, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 718 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 717, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 717, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 446, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 269, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-09-19T04:09:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-09-19T04:09:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 93, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 4, 2024-09-19T04:09:44.994 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 6, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 6, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.995 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:44.996 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 824, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 879260, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 824, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 824, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 879260, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 717, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 1435, 2024-09-19T04:09:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 717, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1435, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-19T04:09:44.999 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 717, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 717, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 717, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 717, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:45.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:45.001 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:45.001 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:45.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-19T04:09:45.001 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:45.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-19T04:09:45.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-19T04:09:45.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:45.004 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:45.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55f617a24b40": { 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-19T04:09:45.006 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-19T04:09:45.007 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-19T04:09:45.007 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:45.007 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:45.007 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:45.007 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:45.007 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:45.007 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:45.007 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:09:45.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:45 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:45.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:45 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:45.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:45 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:09:45.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:45 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:45.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:45 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:45.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:45 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:09:45.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:45 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:09:45.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:45 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:09:45.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:45 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' 2024-09-19T04:09:45.983 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T04:09:45.983 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T04:09:45.983 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-19T04:09:45.983 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-19T04:09:45.983 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T04:09:45.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T04:09:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:09:46.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:46 smithi019 ceph-mon[30938]: pgmap v445: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 384 KiB/s wr, 184 op/s 2024-09-19T04:09:46.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:46 smithi111 ceph-mon[35365]: pgmap v445: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 384 KiB/s wr, 184 op/s 2024-09-19T04:09:46.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:46 smithi196 ceph-mon[35731]: pgmap v445: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 384 KiB/s wr, 184 op/s 2024-09-19T04:09:46.770 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-19T04:09:46.770 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:46.770 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 100, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 95, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 721936, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 57312, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.019151449, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.006248863, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.018653426, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:46.771 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 94, 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.005361966, 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000057042 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 721936, 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 57312 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:46.772 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 5, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 74, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3232, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 91712, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 3, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.007626923, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.003899116, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.003090780, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 73, 2024-09-19T04:09:46.773 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.007134439, 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000097732 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 91712 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:46.774 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.000249364, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000183425, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:46.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.776 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148823, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 27, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-19T04:09:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-09-19T04:09:46.780 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 101, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 707833, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 101, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-19T04:09:46.781 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 101, 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 707833, 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-19T04:09:46.782 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-19T04:09:47.564 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T04:09:47.564 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T04:09:47.564 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T04:09:47.564 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T04:09:47.564 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T04:09:47.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T04:09:47.823 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:09:48.333 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-19T04:09:48.333 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 8327, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 13325, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 3354899, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 9573094, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 32, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 8, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.547711423, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.200308051, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.343301343, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.067175875, 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:48.334 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 13316, 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.752994480, 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000056548 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 14578, 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002298265, 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000157 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 841744, 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 90848 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:48.335 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 3773, 2024-09-19T04:09:48.336 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 4211, 2024-09-19T04:09:48.336 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1062449, 2024-09-19T04:09:48.336 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 6039397, 2024-09-19T04:09:48.336 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 11, 2024-09-19T04:09:48.336 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-09-19T04:09:48.336 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.236747464, 2024-09-19T04:09:48.336 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.078881504, 2024-09-19T04:09:48.337 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.077630525, 2024-09-19T04:09:48.337 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.067640206, 2024-09-19T04:09:48.337 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:48.337 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4212, 2024-09-19T04:09:48.337 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.592580141, 2024-09-19T04:09:48.338 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000140688 2024-09-19T04:09:48.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.338 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:48.338 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1483, 2024-09-19T04:09:48.338 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000302457, 2024-09-19T04:09:48.338 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000203 2024-09-19T04:09:48.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 199232 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 4066, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 4747, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1031138, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 6267291, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 34, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 7, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.298253693, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.113034806, 2024-09-19T04:09:48.339 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.110372152, 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.069088830, 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4751, 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.334105114, 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000070323 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3125, 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000542544, 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000173 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 34000, 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 5136 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4550, 2024-09-19T04:09:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2.144686919, 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000471359 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 5528, 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.231929873, 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000041955 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.342 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 2492, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 2492, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2492, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 36.777863792, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.014758372 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 75, 2024-09-19T04:09:48.343 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 212, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 5, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 207, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 204, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 1, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 1, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 2549, 2024-09-19T04:09:48.344 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 2526, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-19T04:09:48.345 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 5, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 2, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 35, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 3653, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 2452, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 1212, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 14, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 1245 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-19T04:09:48.347 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 31, 2024-09-19T04:09:48.348 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-19T04:09:48.348 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-19T04:09:48.348 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-09-19T04:09:48.348 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 55, 2024-09-19T04:09:48.348 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-19T04:09:48.348 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:48.348 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:48.348 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 1212, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 1212, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 3, 2024-09-19T04:09:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 38, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 6140, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 4621, 2024-09-19T04:09:48.350 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 4621, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 1519, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 3606, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 18, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 15, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 15, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 3, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 1, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 4, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 14046988, 2024-09-19T04:09:48.351 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 16906388, 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2471, 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 4.311874352, 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001744991 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 23, 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 1235, 2024-09-19T04:09:48.352 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 1212, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 18, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 24, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 6, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 212, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 1243, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 1031, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 1, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 1229, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 1228, 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 59088, 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 231712 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 2825, 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 2492, 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 18, 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1210, 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1.076397714, 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000889584 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 31, 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.012937776, 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000417347 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-19T04:09:48.355 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8, 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.003048851, 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000381106 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.357 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-19T04:09:48.358 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 12, 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 19.768222733, 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 1.647351894 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-19T04:09:48.359 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000339597, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000339597 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.360 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 5, 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.005880714, 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001176142 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002156699, 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001078349 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.361 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.362 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 6, 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.031435301, 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005239216 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.363 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001316856, 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000658428 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.364 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 9, 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.021462154, 2024-09-19T04:09:48.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002384683 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1206, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 15.854665397, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.013146488 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 2151, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 2151, 2024-09-19T04:09:48.367 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 161, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:48.368 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:48.369 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 287093, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 101, 2024-09-19T04:09:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 88, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 1, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 168, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:48.371 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 236634, 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2253, 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-19T04:09:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 8418, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 13232756, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 31, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 8418, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8418, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 48.381128284, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005747342 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8418, 2024-09-19T04:09:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 9308 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 8418, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 29, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 8389, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 54, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 461, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 29, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 2500, 2024-09-19T04:09:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 65, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 5221, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 706, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:48.375 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-19T04:09:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-19T04:09:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 93, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 10, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 12, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 12, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 243, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 29 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.378 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 30 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:48.379 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 12, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 5, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 4704, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 1212, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 16162, 2024-09-19T04:09:48.380 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 3974143, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 16162, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 16162, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 3974143, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 8418, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 13364816, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 8418, 2024-09-19T04:09:48.382 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 13364816, 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:48.383 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 8418, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 8418, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 8418, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 8418, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.384 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 1212, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 132060, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 1212, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 29, 2024-09-19T04:09:48.385 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 132060, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x56519dfc8960": { 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 6140, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 12712084, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:48.386 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 6140, 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2471, 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 12712084, 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:48.387 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-19T04:09:48.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:48 smithi111 ceph-mon[35365]: pgmap v446: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 71 KiB/s wr, 60 op/s 2024-09-19T04:09:48.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:48 smithi019 ceph-mon[30938]: pgmap v446: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 71 KiB/s wr, 60 op/s 2024-09-19T04:09:48.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:48 smithi196 ceph-mon[35731]: pgmap v446: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 71 KiB/s wr, 60 op/s 2024-09-19T04:09:49.301 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-19T04:09:49.301 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-19T04:09:49.302 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=31 2024-09-19T04:09:49.302 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-19T04:09:49.302 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T04:09:49.302 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T04:09:49.549 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:09:50.080 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-19T04:09:50.080 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:50.080 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 564, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 563, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 4363821, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 178220, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.052065202, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.018437847, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.015253880, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.013262099, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 562, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.025433779, 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000045255 2024-09-19T04:09:50.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 139, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 132, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 890604, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 67338, 2024-09-19T04:09:50.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.021634468, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.007104543, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.020780895, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000741469, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 131, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.009132970, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000069717 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 753616, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 58368 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 282, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 352, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 8458413, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 275375, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 5, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.033364549, 2024-09-19T04:09:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.009822564, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.011689036, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.009031875, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 350, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.012811306, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000036603 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.085 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 186016 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1447, 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.035354819, 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000024433 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 1233, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 536, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 697, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:50.088 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 466, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-19T04:09:50.089 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:50.090 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-09-19T04:09:50.091 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:50.092 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:50.093 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 6140, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-19T04:09:50.094 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 3, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 13, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 14046988, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 16906388, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 16904367, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 6140 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.095 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 21, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 1233, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 1212, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 16, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 22, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 6, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 1233, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 1240, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 7, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 50372, 2024-09-19T04:09:50.096 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 215328 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.098 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-19T04:09:50.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.101 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-19T04:09:50.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:50.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-19T04:09:50.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 2756, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-19T04:09:50.109 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:50.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:50.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 197337, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 53, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:50.112 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:50.113 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:50.113 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:50.113 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 168, 2024-09-19T04:09:50.113 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:50.113 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:50.113 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 935578, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 5203, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 868, 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 26, 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 4, 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 868, 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 868, 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1.111979970, 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001281082 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 868, 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 869 2024-09-19T04:09:50.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 868, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 868, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 535, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 330, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-19T04:09:50.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 2, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-19T04:09:50.117 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-19T04:09:50.118 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 93, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 4, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 6, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 6, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-19T04:09:50.119 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 981, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 13615821, 2024-09-19T04:09:50.121 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 981, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 981, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 13615821, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:50.122 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 868, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 12712094, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 868, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 12712094, 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:50.123 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 868, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 868, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 868, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 868, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:50.125 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x56442d52ab40": { 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:50.127 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:50.128 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:50.128 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:50.128 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.128 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:50.128 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-19T04:09:50.380 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:50 smithi111 ceph-mon[35365]: pgmap v447: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 KiB/s wr, 30 op/s 2024-09-19T04:09:50.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:50 smithi019 ceph-mon[30938]: pgmap v447: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 KiB/s wr, 30 op/s 2024-09-19T04:09:50.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:50 smithi196 ceph-mon[35731]: pgmap v447: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 KiB/s wr, 30 op/s 2024-09-19T04:09:50.862 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T04:09:50.862 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T04:09:50.862 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-19T04:09:50.862 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-19T04:09:50.862 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T04:09:50.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T04:09:51.109 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:09:51.664 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 28, 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 30, 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 6848, 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 60188, 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 13, 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.007240458, 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000910458, 2024-09-19T04:09:51.665 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.004224138, 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000454476, 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 28, 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.017306138, 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000618076 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 12, 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000001879, 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000156 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 156, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 144, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 774249, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 92540, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 7, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.023096010, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.007071643, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.023406135, 2024-09-19T04:09:51.667 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000301990, 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 144, 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.019736623, 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000137059 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000143, 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000071 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 770672, 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 62176 2024-09-19T04:09:51.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 946, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 641, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 278180, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 331213, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.066088352, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.029718011, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.042989329, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.001470629, 2024-09-19T04:09:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 639, 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.027436656, 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000042936 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1487, 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000256973, 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000172 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 187456 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-19T04:09:51.670 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 40, 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001593971, 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000039849 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-19T04:09:51.671 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000010623, 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000005311 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 1, 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 1, 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000129560, 2024-09-19T04:09:51.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000129560 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 18, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 2, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 16, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:51.673 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 13, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 9, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 1, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 1, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 3, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 19, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 14, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 5, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-19T04:09:51.674 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 5, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:51.675 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 1, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 1, 2024-09-19T04:09:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 21, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 13, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 14, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-19T04:09:51.677 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 3, 2024-09-19T04:09:51.678 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 21, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 3, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-19T04:09:51.679 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 3, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 2, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-09-19T04:09:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4195200, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.003217370, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001608685 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 22, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 28, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 6, 2024-09-19T04:09:51.681 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 21, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 28, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 7, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 18, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 27, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 9, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 1, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 1, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 50120, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 223520 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.682 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 1, 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 1, 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 3, 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-19T04:09:51.683 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.684 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.685 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000129560, 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000129560 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-19T04:09:51.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.688 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.688 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.688 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-19T04:09:51.688 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.688 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.691 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.692 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.694 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 1, 2024-09-19T04:09:51.695 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 0, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 1, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 1, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 152, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:51.696 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:51.698 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 238279, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 78, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:51.699 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 168, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 89669, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 349, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-19T04:09:51.700 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 37, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 11108, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 29, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 37, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 37, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.191996547, 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005189095 2024-09-19T04:09:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 37, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 59 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 37, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 37, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-09-19T04:09:51.702 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 2, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 6, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:51.703 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-19T04:09:51.704 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-19T04:09:51.705 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 93, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 4, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 8, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 8, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 14, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-09-19T04:09:51.706 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 1 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 18 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:51.707 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 1126, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 947984, 2024-09-19T04:09:51.708 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 1126, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 1126, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 947984, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.709 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 37, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 11108, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 37, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 11108, 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:51.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 37, 2024-09-19T04:09:51.711 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 37, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 37, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 37, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:51.712 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:51.713 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55d290336960": { 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 3, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 896, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 3, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 896, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:51.715 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.715 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:51.715 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-19T04:09:52.279 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:52 smithi111 ceph-mon[35365]: pgmap v448: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-09-19T04:09:52.496 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T04:09:52.496 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T04:09:52.496 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-19T04:09:52.497 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=21 2024-09-19T04:09:52.497 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T04:09:52.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T04:09:52.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:52 smithi019 ceph-mon[30938]: pgmap v448: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-09-19T04:09:52.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:52 smithi196 ceph-mon[35731]: pgmap v448: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-09-19T04:09:52.745 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:09:53.345 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-19T04:09:53.345 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:53.345 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 280, 2024-09-19T04:09:53.345 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 280, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 76887, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 88280, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.026317637, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.009364487, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.008213589, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.005892529, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 279, 2024-09-19T04:09:53.346 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.014783719, 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000052988 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:53.347 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 165, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 158, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 767112, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 76991, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.023654608, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.007770259, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.021587121, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.001411470, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 157, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.010109732, 2024-09-19T04:09:53.348 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000064393 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 754544, 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 60768 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 272, 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 345, 2024-09-19T04:09:53.349 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 91216, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 274338, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.034270391, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.011824523, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.008921349, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.009796457, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 343, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.018489302, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000053904 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.350 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 188160 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-19T04:09:53.351 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1137, 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.031034934, 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000027295 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:53.353 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-09-19T04:09:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 1, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-19T04:09:53.355 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:53.356 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-19T04:09:53.357 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:53.358 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-09-19T04:09:53.359 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 3, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-19T04:09:53.360 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4195200, 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 4194892, 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 3 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 3, 2024-09-19T04:09:53.361 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 3, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 1, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 1, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 0, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 0, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 0, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 48392, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 215328 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.362 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.363 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-19T04:09:53.364 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.365 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.366 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.367 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-19T04:09:53.369 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.370 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.372 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.374 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 1, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 0, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 1463, 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:53.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:53.376 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:53.377 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 189239, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 48, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:53.378 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 168, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 8512, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 13, 2024-09-19T04:09:53.379 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 598, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 21, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 3, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 598, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 598, 2024-09-19T04:09:53.380 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.274995044, 2024-09-19T04:09:53.381 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000459857 2024-09-19T04:09:53.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.381 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-19T04:09:53.381 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-19T04:09:53.381 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 598, 2024-09-19T04:09:53.384 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 599 2024-09-19T04:09:53.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.384 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 598, 2024-09-19T04:09:53.384 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 598, 2024-09-19T04:09:53.384 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2024-09-19T04:09:53.384 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-19T04:09:53.384 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 320, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 276, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:53.385 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 1, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-19T04:09:53.386 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-19T04:09:53.387 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 93, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 4, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 5, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 5, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2024-09-19T04:09:53.388 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-19T04:09:53.389 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 713, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 862156, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 713, 2024-09-19T04:09:53.390 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 713, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 862156, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:53.391 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 598, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 901, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 598, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 901, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 598, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 598, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 598, 2024-09-19T04:09:53.393 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 598, 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:53.394 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-19T04:09:53.395 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x5614f5156b40": { 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-19T04:09:53.396 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-19T04:09:53.397 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-19T04:09:53.397 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.397 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-19T04:09:53.397 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-19T04:09:54.098 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T04:09:54.098 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T04:09:54.098 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-19T04:09:54.098 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-19T04:09:54.098 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T04:09:54.098 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T04:09:54.357 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:09:54.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:54 smithi111 ceph-mon[35365]: pgmap v449: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-09-19T04:09:54.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:54 smithi196 ceph-mon[35731]: pgmap v449: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-09-19T04:09:54.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:54 smithi019 ceph-mon[30938]: pgmap v449: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-09-19T04:09:54.935 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 26, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 28, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 6131, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 83535, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 15, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.006662994, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.000596088, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.003929805, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000454766, 2024-09-19T04:09:54.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 26, 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014943976, 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000574768 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 10, 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000001280, 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000128 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.937 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 160, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 148, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 777790, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 75752, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 5, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.021636553, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.007240976, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.022795672, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000400172, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 147, 2024-09-19T04:09:54.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.019893567, 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000135330 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 774000, 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 63648 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.939 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 954, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 647, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 281428, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 303312, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.070505698, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.026849607, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.052407776, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.001814932, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 646, 2024-09-19T04:09:54.940 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.029990660, 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000046425 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1496, 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000268343, 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000179 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 189696 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-19T04:09:54.941 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 41, 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001432783, 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000034945 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T04:09:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000018152, 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000009076 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.943 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 18, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 2, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-09-19T04:09:54.944 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 9, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 19, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 14, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 5, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-19T04:09:54.945 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 5, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:54.946 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:09:54.947 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 21, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 13, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 14, 2024-09-19T04:09:54.948 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-19T04:09:54.949 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 3, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 21, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 3, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-19T04:09:54.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 3, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195200, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-19T04:09:54.951 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.003169974, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001584987 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 22, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 28, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 6, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 21, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 28, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 7, 2024-09-19T04:09:54.952 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 18, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 27, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 9, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50312, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:54.953 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 2, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:54.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.955 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.960 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.962 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:54.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.962 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.963 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-19T04:09:54.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.963 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.964 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:54.964 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.964 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.964 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-19T04:09:54.965 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-19T04:09:54.967 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 1, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 46, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:54.968 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:54.969 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:54.970 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 230263, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 76, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:54.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 168, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 89293, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 346, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-19T04:09:54.972 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 37, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 11108, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 37, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 37, 2024-09-19T04:09:54.973 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.196995700, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005324208 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 37, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 59 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 37, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 37, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 2, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 6, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-19T04:09:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-19T04:09:54.976 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-19T04:09:54.977 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 93, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 4, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 8, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 8, 2024-09-19T04:09:54.978 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 14, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 18 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1136, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 952772, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1136, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1136, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 952772, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.981 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 37, 2024-09-19T04:09:54.982 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 11108, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 37, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 11108, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:54.983 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 37, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 37, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 37, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 37, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:54.984 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x56046e49e960": { 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 896, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 896, 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:54.987 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:54.987 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:54.987 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.987 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:54.987 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T04:09:55.820 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T04:09:55.820 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T04:09:55.821 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-19T04:09:55.821 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=21 2024-09-19T04:09:55.821 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T04:09:55.821 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T04:09:56.067 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:09:56.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:56 smithi196 ceph-mon[35731]: pgmap v450: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-19T04:09:56.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:56 smithi111 ceph-mon[35365]: pgmap v450: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-19T04:09:56.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:56 smithi019 ceph-mon[30938]: pgmap v450: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-19T04:09:56.931 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T04:09:56.931 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:56.931 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 963, 2024-09-19T04:09:56.931 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 585, 2024-09-19T04:09:56.931 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 285660, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 154232, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.066099930, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.023503480, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.048929811, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.002770134, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 583, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.025520580, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000043774 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1494, 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000262477, 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000175 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 160, 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 152, 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 767338, 2024-09-19T04:09:56.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 100750, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 10, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.025225872, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.008047989, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.026241576, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000178131, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 151, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014888419, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000098598 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 21, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000010952, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000521 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 763408, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 64928 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 25, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 94, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 8177, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 247201, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 10, 2024-09-19T04:09:56.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.009685104, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.003298710, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.005150374, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000379230, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 93, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.021914884, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000235643 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4, 2024-09-19T04:09:56.936 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000304, 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000076 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 189872 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T04:09:56.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 42, 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001433598, 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000034133 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000015139, 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000007569 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 3, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 2, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000322722, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000161361 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 1, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:56.939 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 18, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 2, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 9, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 1, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 1, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 22, 2024-09-19T04:09:56.940 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 15, 2024-09-19T04:09:56.941 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-19T04:09:56.941 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 6, 2024-09-19T04:09:56.941 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:56.941 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:56.941 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-19T04:09:56.941 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 5, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-19T04:09:56.942 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 1, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:09:56.943 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 1, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 1 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 19, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 12, 2024-09-19T04:09:56.944 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 14, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-19T04:09:56.945 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 3, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 21, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 5, 2024-09-19T04:09:56.946 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 5, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-19T04:09:56.947 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195734, 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4, 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.006767841, 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001691960 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 22, 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 28, 2024-09-19T04:09:56.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 6, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 21, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 27, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 6, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 18, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 27, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 9, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 1, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 2, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 1, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50292, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.949 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 4, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 3, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 7, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-09-19T04:09:56.950 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000201905, 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000201905 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000120817, 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000120817 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.953 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.956 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:56.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-19T04:09:56.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 1, 2024-09-19T04:09:56.962 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 1, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 1, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 157, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:56.963 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:56.964 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:56.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 253799, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 76, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:56.966 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 168, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 89701, 2024-09-19T04:09:56.967 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 353, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 41, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 11943, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 28, 2024-09-19T04:09:56.968 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 41, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 41, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.212995346, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005195008 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 41, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 64 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 41, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-09-19T04:09:56.969 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 41, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 4, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 7, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-09-19T04:09:56.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:56.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-19T04:09:56.972 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 93, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 4, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 9, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 16, 2024-09-19T04:09:56.973 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 18 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1144, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 947979, 2024-09-19T04:09:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1144, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1144, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 947979, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:56.976 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 41, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 11943, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 41, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 11943, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 41, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 41, 2024-09-19T04:09:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 41, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 41, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:56.979 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55bec190a960": { 2024-09-19T04:09:56.980 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 5, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1430, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 5, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 4, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1430, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:56.981 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:56.982 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:56.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:56.982 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T04:09:57.835 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T04:09:57.835 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T04:09:57.835 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-19T04:09:57.835 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=19 2024-09-19T04:09:57.836 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T04:09:57.836 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T04:09:58.090 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:09:58.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:09:58 smithi196 ceph-mon[35731]: pgmap v451: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:09:58.643 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 973, 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 599, 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 283245, 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 144789, 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.069564014, 2024-09-19T04:09:58.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.024557768, 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.053205151, 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.002285609, 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 597, 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.039453963, 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000066087 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1511, 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000308308, 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000204 2024-09-19T04:09:58.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 174, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 159, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 783954, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 120876, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 11, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.025131799, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.007807175, 2024-09-19T04:09:58.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.024520811, 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000401842, 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 158, 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.019198045, 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000121506 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 24, 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000003862, 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000160 2024-09-19T04:09:58.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 777712, 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 63008 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 19, 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 87, 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 6955, 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 227372, 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-09-19T04:09:58.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.009185537, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.003405421, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.004650978, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000334237, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 86, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014558976, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000169290 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000161, 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000080 2024-09-19T04:09:58.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 191312 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 49, 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001437981, 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000029346 2024-09-19T04:09:58.650 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000014588, 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000007294 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 4, 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 3, 2024-09-19T04:09:58.651 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 8.982028815, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 2.994009605 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 1, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-19T04:09:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 18, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 2, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 8, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 28, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 14, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 1, 2024-09-19T04:09:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 4, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 3, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-19T04:09:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 5, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:09:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 1, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 1, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 2, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 1, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-19T04:09:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 20, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 12, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 13, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:09:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 3, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 21, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-19T04:09:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 3, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 3, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-19T04:09:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195200, 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002921028, 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001460514 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-19T04:09:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 22, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 28, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 6, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 21, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 28, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 7, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 18, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 26, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 8, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 1, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 1, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50400, 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-19T04:09:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 7, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 4, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 3, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-19T04:09:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3, 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 8.982028815, 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 2.994009605 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-19T04:09:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-19T04:09:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.667 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:09:58 smithi111 ceph-mon[35365]: pgmap v451: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:09:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-19T04:09:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 1, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-09-19T04:09:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 2, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 2, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 173, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:09:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:09:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:09:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 254375, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 84, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-19T04:09:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8872, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 168, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 89317, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 349, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-19T04:09:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 41, 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 11932, 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 41, 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 41, 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.210995391, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005146229 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 41, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 68 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 41, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 41, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-19T04:09:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 2, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 6, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-09-19T04:09:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:09:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-19T04:09:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 93, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 4, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 9, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-19T04:09:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 21, 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 3 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 18 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-19T04:09:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1162, 2024-09-19T04:09:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 959199, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1162, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1162, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 959199, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 41, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 11932, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 41, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 11932, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 41, 2024-09-19T04:09:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 41, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 41, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 41, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:09:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55b4ab3dc960": { 2024-09-19T04:09:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 896, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 896, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:09:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:09:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:09:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:09:58.694 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T04:09:58.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:09:58 smithi019 ceph-mon[30938]: pgmap v451: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T04:09:59.498 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T04:09:59.498 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T04:09:59.498 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-19T04:09:59.498 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=20 2024-09-19T04:09:59.498 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T04:09:59.498 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T04:09:59.743 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:10:00.263 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 741, 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 741, 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 215361, 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 234821, 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.072170347, 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.023715658, 2024-09-19T04:10:00.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.019769715, 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.022577694, 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 740, 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.035123854, 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000047464 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 119, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 112, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 756540, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 63082, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 3, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.020133248, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.006794853, 2024-09-19T04:10:00.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.020932880, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000050760, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 111, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.007699542, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000069365 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 755984, 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 62384 2024-09-19T04:10:00.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 77, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 428, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 192062, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.005288520, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.003207057, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.001118948, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000035628, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 75, 2024-09-19T04:10:00.268 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.005650296, 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000075337 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 191744 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-19T04:10:00.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1296, 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.037030674, 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000028573 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.270 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-19T04:10:00.271 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-19T04:10:00.272 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-19T04:10:00.273 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-19T04:10:00.274 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-19T04:10:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-19T04:10:00.276 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-19T04:10:00.277 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 3, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195200, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 4194892, 2024-09-19T04:10:00.278 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 3 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 3, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 3, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 1, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 1, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 0, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 0, 2024-09-19T04:10:00.279 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50364, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215328 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-09-19T04:10:00.280 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-19T04:10:00.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.284 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-19T04:10:00.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.285 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.285 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-19T04:10:00.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.285 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.286 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-19T04:10:00.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-19T04:10:00.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.290 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-19T04:10:00.292 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 1, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-19T04:10:00.293 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1500, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T04:10:00.294 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T04:10:00.295 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-19T04:10:00.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 181159, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 44, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8872, 2024-09-19T04:10:00.297 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 168, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 8512, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 13, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-19T04:10:00.298 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 740, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 740, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 740, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.358992094, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000485124 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 740, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 741 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.299 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 740, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 740, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 455, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 283, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-19T04:10:00.300 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T04:10:00.301 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-19T04:10:00.302 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 93, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 4, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 4, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 4, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-09-19T04:10:00.303 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-19T04:10:00.304 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 858, 2024-09-19T04:10:00.305 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 886077, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 858, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 858, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 886077, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.306 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 740, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 901, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 740, 2024-09-19T04:10:00.307 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 901, 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:10:00.308 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 740, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 740, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 740, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 740, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-19T04:10:00.309 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.310 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x560e5bb9eb40": { 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T04:10:00.311 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T04:10:00.312 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T04:10:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:00 smithi196 ceph-mon[35731]: pgmap v452: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:10:00.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:00 smithi196 ceph-mon[35731]: overall HEALTH_OK 2024-09-19T04:10:00.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:00 smithi111 ceph-mon[35365]: pgmap v452: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:10:00.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:00 smithi111 ceph-mon[35365]: overall HEALTH_OK 2024-09-19T04:10:00.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:00 smithi019 ceph-mon[30938]: pgmap v452: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T04:10:00.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:00 smithi019 ceph-mon[30938]: overall HEALTH_OK 2024-09-19T04:10:01.064 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T04:10:01.064 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T04:10:01.064 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-19T04:10:01.064 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-19T04:10:01.064 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T04:10:01.073 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T04:10:01.074 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T04:10:01.074 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:10:01.097 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-19T04:10:01.097 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:10:01.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T04:10:01.470 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 -- 172.21.15.19:0/290246965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 msgr2=0x7f2954100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 --2- 172.21.15.19:0/290246965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 0x7f2954100560 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f293c009900 tx=0x7f293c02a3f0 comp rx=0 tx=0).stop 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 -- 172.21.15.19:0/290246965 shutdown_connections 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 --2- 172.21.15.19:0/290246965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2954100aa0 0x7f295410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 --2- 172.21.15.19:0/290246965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 0x7f2954100560 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 --2- 172.21.15.19:0/290246965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2954106b80 0x7f2954106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 -- 172.21.15.19:0/290246965 >> 172.21.15.19:0/290246965 conn(0x7f2954077230 msgr2=0x7f29540fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 -- 172.21.15.19:0/290246965 shutdown_connections 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.967+0000 7f2959658640 1 -- 172.21.15.19:0/290246965 wait complete. 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2959658640 1 Processor -- start 2024-09-19T04:10:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2959658640 1 -- start start 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2959658640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 0x7f295419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2959658640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2954100aa0 0x7f295419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2959658640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2954106b80 0x7f295419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2959658640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2954112050 con 0x7f2954100100 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2959658640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2954111ed0 con 0x7f2954100aa0 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2959658640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29541121d0 con 0x7f2954106b80 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 0x7f295419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 0x7f295419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54586/0 (socket says 172.21.15.19:54586) 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.968+0000 7f2952ffd640 1 -- 172.21.15.19:0/2562585783 learned_addr learned my addr 172.21.15.19:0/2562585783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f29537fe640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2954106b80 0x7f295419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f29527fc640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2954100aa0 0x7f295419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f2952ffd640 1 -- 172.21.15.19:0/2562585783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2954106b80 msgr2=0x7f295419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f2952ffd640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2954106b80 0x7f295419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f2952ffd640 1 -- 172.21.15.19:0/2562585783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2954100aa0 msgr2=0x7f295419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f2952ffd640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2954100aa0 0x7f295419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f2952ffd640 1 -- 172.21.15.19:0/2562585783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295419f590 con 0x7f2954100100 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f29537fe640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2954106b80 0x7f295419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f29527fc640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2954100aa0 0x7f295419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f2952ffd640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 0x7f295419a5e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f294c00c920 tx=0x7f294c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f294bfff640 1 -- 172.21.15.19:0/2562585783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c007c70 con 0x7f2954100100 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f294bfff640 1 -- 172.21.15.19:0/2562585783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c007e10 con 0x7f2954100100 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295419f880 con 0x7f2954100100 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f294bfff640 1 -- 172.21.15.19:0/2562585783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c018890 con 0x7f2954100100 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.969+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29541a7160 con 0x7f2954100100 2024-09-19T04:10:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.970+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2954100560 con 0x7f2954100100 2024-09-19T04:10:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.971+0000 7f294bfff640 1 -- 172.21.15.19:0/2562585783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f294c018a70 con 0x7f2954100100 2024-09-19T04:10:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.972+0000 7f294bfff640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29400a32c0 0x7f29400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.972+0000 7f294bfff640 1 -- 172.21.15.19:0/2562585783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f294c0cfe80 con 0x7f2954100100 2024-09-19T04:10:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.972+0000 7f29527fc640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29400a32c0 0x7f29400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.972+0000 7f29527fc640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29400a32c0 0x7f29400a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f293c000c00 tx=0x7f293c032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:01.976+0000 7f294bfff640 1 -- 172.21.15.19:0/2562585783 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f294c014030 con 0x7f2954100100 2024-09-19T04:10:02.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.162+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f295419b940 con 0x7f2954100100 2024-09-19T04:10:02.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.163+0000 7f294bfff640 1 -- 172.21.15.19:0/2562585783 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+590 (secure 0 0 0) 0x7f294c09c800 con 0x7f2954100100 2024-09-19T04:10:02.165 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1292798638 2024-09-20T03:57:40.326582+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2229567042 2024-09-20T03:57:40.326582+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2229567042 2024-09-20T03:57:40.326582+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2828287970 2024-09-20T03:57:40.326582+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/569607576 2024-09-20T03:57:15.012251+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/748113766 2024-09-20T03:57:15.012251+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3673052449 2024-09-20T03:57:15.012251+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3967546781 2024-09-20T03:57:40.326582+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3897870084 2024-09-20T03:57:15.012251+0000 2024-09-19T04:10:02.166 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3897870084 2024-09-20T03:57:15.012251+0000 2024-09-19T04:10:02.167 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-09-19T04:10:02.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29400a32c0 msgr2=0x7f29400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29400a32c0 0x7f29400a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f293c000c00 tx=0x7f293c032d50 comp rx=0 tx=0).stop 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 msgr2=0x7f295419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 0x7f295419a5e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f294c00c920 tx=0x7f294c00cde0 comp rx=0 tx=0).stop 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 shutdown_connections 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f29400a32c0 0x7f29400a5780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2954106b80 0x7f295419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2954100aa0 0x7f295419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 --2- 172.21.15.19:0/2562585783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2954100100 0x7f295419a5e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 >> 172.21.15.19:0/2562585783 conn(0x7f2954077230 msgr2=0x7f295410dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 shutdown_connections 2024-09-19T04:10:02.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:02.165+0000 7f2959658640 1 -- 172.21.15.19:0/2562585783 wait complete. 2024-09-19T04:10:02.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:02 smithi019 ceph-mon[30938]: pgmap v453: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T04:10:02.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:02 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/2562585783' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:10:02.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:02 smithi111 ceph-mon[35365]: pgmap v453: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T04:10:02.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:02 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/2562585783' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:10:02.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:02 smithi196 ceph-mon[35731]: pgmap v453: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T04:10:02.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:02 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/2562585783' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T04:10:02.927 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T04:10:02.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:10:02.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T04:10:02.975 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T04:10:02.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:10:02.975 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:10:03.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:10:03.009 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-09-19T04:10:03.066 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:10:03.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:10:03.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:10:03.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:10:03.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-09-19T04:10:03.229 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T04:10:03.239 INFO:tasks.cephadm:Teardown begin 2024-09-19T04:10:03.239 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:10:03.240 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:10:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 -- 172.21.15.19:0/4209147785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018101de0 msgr2=0x7f60181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 --2- 172.21.15.19:0/4209147785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018101de0 0x7f60181021c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f600c009900 tx=0x7f600c02a410 comp rx=0 tx=0).stop 2024-09-19T04:10:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 -- 172.21.15.19:0/4209147785 shutdown_connections 2024-09-19T04:10:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 --2- 172.21.15.19:0/4209147785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6018106f10 0x7f601810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 --2- 172.21.15.19:0/4209147785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6018102790 0x7f60181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 --2- 172.21.15.19:0/4209147785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018101de0 0x7f60181021c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 -- 172.21.15.19:0/4209147785 >> 172.21.15.19:0/4209147785 conn(0x7f60180fd8c0 msgr2=0x7f60180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 -- 172.21.15.19:0/4209147785 shutdown_connections 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 -- 172.21.15.19:0/4209147785 wait complete. 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 Processor -- start 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.977+0000 7f601ee53640 1 -- start start 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601ee53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6018101de0 0x7f60181984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601ee53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018102790 0x7f60181989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601ee53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6018106f10 0x7f601819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601ee53640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6018111f70 con 0x7f6018102790 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601ee53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6018111df0 con 0x7f6018101de0 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601ee53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60181120f0 con 0x7f6018106f10 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018102790 0x7f60181989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018102790 0x7f60181989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54598/0 (socket says 172.21.15.19:54598) 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 -- 172.21.15.19:0/443654817 learned_addr learned my addr 172.21.15.19:0/443654817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601cbef640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6018101de0 0x7f60181984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601d3f0640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6018106f10 0x7f601819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 -- 172.21.15.19:0/443654817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6018106f10 msgr2=0x7f601819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6018106f10 0x7f601819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 -- 172.21.15.19:0/443654817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6018101de0 msgr2=0x7f60181984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6018101de0 0x7f60181984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 -- 172.21.15.19:0/443654817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601819d460 con 0x7f6018102790 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f601d3f0640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6018106f10 0x7f601819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.978+0000 7f6017fff640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018102790 0x7f60181989f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f600400d770 tx=0x7f600400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.979+0000 7f6015ffb640 1 -- 172.21.15.19:0/443654817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004017070 con 0x7f6018102790 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.979+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601819d750 con 0x7f6018102790 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.979+0000 7f6015ffb640 1 -- 172.21.15.19:0/443654817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6004005500 con 0x7f6018102790 2024-09-19T04:10:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.979+0000 7f6015ffb640 1 -- 172.21.15.19:0/443654817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004004e00 con 0x7f6018102790 2024-09-19T04:10:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.979+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60181a5030 con 0x7f6018102790 2024-09-19T04:10:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.980+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60181021c0 con 0x7f6018102790 2024-09-19T04:10:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.981+0000 7f6015ffb640 1 -- 172.21.15.19:0/443654817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6004004fa0 con 0x7f6018102790 2024-09-19T04:10:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.982+0000 7f6015ffb640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f60080a3240 0x7f60080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.982+0000 7f601cbef640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f60080a3240 0x7f60080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.982+0000 7f6015ffb640 1 -- 172.21.15.19:0/443654817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f60040cf790 con 0x7f6018102790 2024-09-19T04:10:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.982+0000 7f601cbef640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f60080a3240 0x7f60080a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f600c009230 tx=0x7f600c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:03.986+0000 7f6015ffb640 1 -- 172.21.15.19:0/443654817 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f600409c0b0 con 0x7f6018102790 2024-09-19T04:10:04.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.204+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f601810a150 con 0x7f6018102790 2024-09-19T04:10:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.204+0000 7f6015ffb640 1 -- 172.21.15.19:0/443654817 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7f600409b9b0 con 0x7f6018102790 2024-09-19T04:10:04.207 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:10:04.207 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:10:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:10:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f60080a3240 msgr2=0x7f60080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f60080a3240 0x7f60080a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f600c009230 tx=0x7f600c0058d0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018102790 msgr2=0x7f60181989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018102790 0x7f60181989f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f600400d770 tx=0x7f600400dc30 comp rx=0 tx=0).stop 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 shutdown_connections 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f60080a3240 0x7f60080a5700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6018106f10 0x7f601819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018102790 0x7f60181989f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 --2- 172.21.15.19:0/443654817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6018101de0 0x7f60181984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 >> 172.21.15.19:0/443654817 conn(0x7f60180fd8c0 msgr2=0x7f60180fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 shutdown_connections 2024-09-19T04:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:04.207+0000 7f601ee53640 1 -- 172.21.15.19:0/443654817 wait complete. 2024-09-19T04:10:04.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:04 smithi019 ceph-mon[30938]: pgmap v454: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T04:10:04.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:04 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/443654817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:10:04.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:04 smithi111 ceph-mon[35365]: pgmap v454: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T04:10:04.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:04 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/443654817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:10:04.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:04 smithi196 ceph-mon[35731]: pgmap v454: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T04:10:04.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:04 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/443654817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:10:04.923 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T04:10:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 -- 172.21.15.19:0/1660890089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094109130 msgr2=0x7f5094110cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 --2- 172.21.15.19:0/1660890089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094109130 0x7f5094110cb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f508c009ca0 tx=0x7f508c02a8c0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 -- 172.21.15.19:0/1660890089 shutdown_connections 2024-09-19T04:10:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 --2- 172.21.15.19:0/1660890089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094109130 0x7f5094110cb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 --2- 172.21.15.19:0/1660890089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50941049b0 0x7f5094108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 --2- 172.21.15.19:0/1660890089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5094104000 0x7f50941043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 -- 172.21.15.19:0/1660890089 >> 172.21.15.19:0/1660890089 conn(0x7f50940ffa20 msgr2=0x7f5094101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 -- 172.21.15.19:0/1660890089 shutdown_connections 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.625+0000 7f509a460640 1 -- 172.21.15.19:0/1660890089 wait complete. 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f509a460640 1 Processor -- start 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f509a460640 1 -- start start 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f509a460640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094104000 0x7f509419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f509a460640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50941049b0 0x7f509419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f509a460640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5094109130 0x7f509419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f509a460640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50941141b0 con 0x7f5094104000 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f509a460640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5094114030 con 0x7f50941049b0 2024-09-19T04:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f509a460640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5094114330 con 0x7f5094109130 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.626+0000 7f5093fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094104000 0x7f509419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f5093fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094104000 0x7f509419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54628/0 (socket says 172.21.15.19:54628) 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f5093fff640 1 -- 172.21.15.19:0/3259287194 learned_addr learned my addr 172.21.15.19:0/3259287194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f50937fe640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50941049b0 0x7f509419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f50989fd640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5094109130 0x7f509419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f5093fff640 1 -- 172.21.15.19:0/3259287194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5094109130 msgr2=0x7f509419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f5093fff640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5094109130 0x7f509419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f5093fff640 1 -- 172.21.15.19:0/3259287194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50941049b0 msgr2=0x7f509419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f5093fff640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50941049b0 0x7f509419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f5093fff640 1 -- 172.21.15.19:0/3259287194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509419f580 con 0x7f5094104000 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f5093fff640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094104000 0x7f509419a5d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f508400b4a0 tx=0x7f508400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f50917fa640 1 -- 172.21.15.19:0/3259287194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084015020 con 0x7f5094104000 2024-09-19T04:10:05.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509419f870 con 0x7f5094104000 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f50917fa640 1 -- 172.21.15.19:0/3259287194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5084011d10 con 0x7f5094104000 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.627+0000 7f50917fa640 1 -- 172.21.15.19:0/3259287194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508400f2e0 con 0x7f5094104000 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.628+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50941a7150 con 0x7f5094104000 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.629+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50941043e0 con 0x7f5094104000 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.630+0000 7f50917fa640 1 -- 172.21.15.19:0/3259287194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5084018020 con 0x7f5094104000 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.631+0000 7f50917fa640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f507c0a3240 0x7f507c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.631+0000 7f50937fe640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f507c0a3240 0x7f507c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.631+0000 7f50917fa640 1 -- 172.21.15.19:0/3259287194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f50840cf550 con 0x7f5094104000 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.631+0000 7f50937fe640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f507c0a3240 0x7f507c0a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f509410aae0 tx=0x7f508800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:05.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.635+0000 7f50917fa640 1 -- 172.21.15.19:0/3259287194 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f508401f020 con 0x7f5094104000 2024-09-19T04:10:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.851+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f509410c310 con 0x7f5094104000 2024-09-19T04:10:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.851+0000 7f50917fa640 1 -- 172.21.15.19:0/3259287194 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11980 (secure 0 0 0) 0x7f508409bed0 con 0x7f5094104000 2024-09-19T04:10:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:10:05.854 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"btime":"2024-09-19T04:07:03:950472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/243568683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":243568683},{"type":"v1","addr":"172.21.15.19:6835","nonce":243568683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2162743100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2162743100},{"type":"v1","addr":"172.21.15.19:6837","nonce":2162743100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T04:04:40.077653+0000","modified":"2024-09-19T04:07:03.493755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24454,"mds_3":24452,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14574":{"gid":14574,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/821554910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":821554910},{"type":"v1","addr":"172.21.15.19:6839","nonce":821554910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/733319067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":733319067},{"type":"v1","addr":"172.21.15.19:6841","nonce":733319067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/3618359356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3618359356},{"type":"v1","addr":"172.21.15.111:6833","nonce":3618359356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2394579385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2394579385},{"type":"v1","addr":"172.21.15.196:6833","nonce":2394579385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/3717777626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":3717777626},{"type":"v1","addr":"172.21.15.111:6835","nonce":3717777626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":19,"addr":"172.21.15.196:6835/1584544131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1584544131},{"type":"v1","addr":"172.21.15.196:6835","nonce":1584544131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.111:6837/1580295791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1580295791},{"type":"v1","addr":"172.21.15.111:6837","nonce":1580295791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/1041250110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1041250110},{"type":"v1","addr":"172.21.15.196:6837","nonce":1041250110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.196:6839/1173261295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1173261295},{"type":"v1","addr":"172.21.15.196:6839","nonce":1173261295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6839/528342006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":528342006},{"type":"v1","addr":"172.21.15.111:6839","nonce":528342006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24448,24452,24454,24469,24476]},"id":1}]} 2024-09-19T04:10:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-09-19T04:10:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.853+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f507c0a3240 msgr2=0x7f507c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f507c0a3240 0x7f507c0a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f509410aae0 tx=0x7f508800a2d0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094104000 msgr2=0x7f509419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094104000 0x7f509419a5d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f508400b4a0 tx=0x7f508400b8a0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 shutdown_connections 2024-09-19T04:10:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f507c0a3240 0x7f507c0a5700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5094109130 0x7f509419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50941049b0 0x7f509419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 --2- 172.21.15.19:0/3259287194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5094104000 0x7f509419a5d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:05.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 >> 172.21.15.19:0/3259287194 conn(0x7f50940ffa20 msgr2=0x7f5094100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:05.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 shutdown_connections 2024-09-19T04:10:05.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:05.854+0000 7f509a460640 1 -- 172.21.15.19:0/3259287194 wait complete. 2024-09-19T04:10:06.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:06 smithi019 ceph-mon[30938]: pgmap v455: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:10:06.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:06 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3259287194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:10:06.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T04:10:06.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:06 smithi111 ceph-mon[35365]: pgmap v455: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:10:06.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:06 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3259287194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:10:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:06 smithi196 ceph-mon[35731]: pgmap v455: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T04:10:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:06 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3259287194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T04:10:06.881 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 -- 172.21.15.19:0/2302428193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154102d30 msgr2=0x7fe154103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 --2- 172.21.15.19:0/2302428193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154102d30 0x7fe154103110 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe144009930 tx=0x7fe14402a420 comp rx=0 tx=0).stop 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 -- 172.21.15.19:0/2302428193 shutdown_connections 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 --2- 172.21.15.19:0/2302428193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1541082a0 0x7fe15410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 --2- 172.21.15.19:0/2302428193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1541036e0 0x7fe154107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 --2- 172.21.15.19:0/2302428193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154102d30 0x7fe154103110 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 -- 172.21.15.19:0/2302428193 >> 172.21.15.19:0/2302428193 conn(0x7fe1540fc7d0 msgr2=0x7fe1540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 -- 172.21.15.19:0/2302428193 shutdown_connections 2024-09-19T04:10:07.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.357+0000 7fe159b4f640 1 -- 172.21.15.19:0/2302428193 wait complete. 2024-09-19T04:10:07.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.358+0000 7fe159b4f640 1 Processor -- start 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.358+0000 7fe159b4f640 1 -- start start 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.358+0000 7fe159b4f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe154102d30 0x7fe1541985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.358+0000 7fe159b4f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541036e0 0x7fe154198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe159b4f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1541082a0 0x7fe15419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe159b4f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541120a0 con 0x7fe1541036e0 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe159b4f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe154111f20 con 0x7fe154102d30 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe159b4f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe154112220 con 0x7fe1541082a0 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe14affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541036e0 0x7fe154198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe14affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541036e0 0x7fe154198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54646/0 (socket says 172.21.15.19:54646) 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe1537fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe154102d30 0x7fe1541985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe14affd640 1 -- 172.21.15.19:0/570718379 learned_addr learned my addr 172.21.15.19:0/570718379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:07.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe153fff640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1541082a0 0x7fe15419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe14affd640 1 -- 172.21.15.19:0/570718379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1541082a0 msgr2=0x7fe15419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe14affd640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1541082a0 0x7fe15419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe14affd640 1 -- 172.21.15.19:0/570718379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe154102d30 msgr2=0x7fe1541985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe14affd640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe154102d30 0x7fe1541985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe14affd640 1 -- 172.21.15.19:0/570718379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15419d590 con 0x7fe1541036e0 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.359+0000 7fe153fff640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1541082a0 0x7fe15419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.360+0000 7fe14affd640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541036e0 0x7fe154198b20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00d750 tx=0x7fe13c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.360+0000 7fe1517fa640 1 -- 172.21.15.19:0/570718379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c017070 con 0x7fe1541036e0 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.360+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15419d880 con 0x7fe1541036e0 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.360+0000 7fe1517fa640 1 -- 172.21.15.19:0/570718379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c005500 con 0x7fe1541036e0 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.360+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541a5160 con 0x7fe1541036e0 2024-09-19T04:10:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.360+0000 7fe1517fa640 1 -- 172.21.15.19:0/570718379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c004e00 con 0x7fe1541036e0 2024-09-19T04:10:07.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.361+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe134005290 con 0x7fe1541036e0 2024-09-19T04:10:07.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.362+0000 7fe1517fa640 1 -- 172.21.15.19:0/570718379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe13c009c30 con 0x7fe1541036e0 2024-09-19T04:10:07.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.363+0000 7fe1517fa640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe1400a32c0 0x7fe1400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:07.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.363+0000 7fe1517fa640 1 -- 172.21.15.19:0/570718379 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7fe13c0cf820 con 0x7fe1541036e0 2024-09-19T04:10:07.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.363+0000 7fe1537fe640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe1400a32c0 0x7fe1400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:07.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.364+0000 7fe1537fe640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe1400a32c0 0x7fe1400a5780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe144009900 tx=0x7fe1440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:07.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.370+0000 7fe1517fa640 1 -- 172.21.15.19:0/570718379 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe13c09bbd0 con 0x7fe1541036e0 2024-09-19T04:10:07.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.549+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe134005850 con 0x7fe1541036e0 2024-09-19T04:10:07.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.551+0000 7fe1517fa640 1 -- 172.21.15.19:0/570718379 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24842 (secure 0 0 0) 0x7fe13c09fed0 con 0x7fe1541036e0 2024-09-19T04:10:07.554 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-19T04:10:07.554 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":93,"fsid":"0518de98-763b-11ef-bceb-c7b262605968","created":"2024-09-19T03:56:42.596342+0000","modified":"2024-09-19T04:06:42.902449+0000","last_up_change":"2024-09-19T04:03:48.439378+0000","last_in_change":"2024-09-19T04:03:24.861444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T03:59:58.845133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:04:36.009364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:04:38.289030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:04:44.768678+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ab98d168-5199-4c3f-a220-00c832addff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6803","nonce":2401652913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6805","nonce":2401652913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6809","nonce":2401652913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2401652913},{"type":"v1","addr":"172.21.15.19:6807","nonce":2401652913}]},"public_addr":"172.21.15.19:6803/2401652913","cluster_addr":"172.21.15.19:6805/2401652913","heartbeat_back_addr":"172.21.15.19:6809/2401652913","heartbeat_front_addr":"172.21.15.19:6807/2401652913","state":["exists","up"]},{"osd":1,"uuid":"66cb950f-6aee-4140-8411-0e9deaf594fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6801","nonce":3201221054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6803","nonce":3201221054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6807","nonce":3201221054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3201221054},{"type":"v1","addr":"172.21.15.111:6805","nonce":3201221054}]},"public_addr":"172.21.15.111:6801/3201221054","cluster_addr":"172.21.15.111:6803/3201221054","heartbeat_back_addr":"172.21.15.111:6807/3201221054","heartbeat_front_addr":"172.21.15.111:6805/3201221054","state":["exists","up"]},{"osd":2,"uuid":"d2ad4ffa-1018-4519-96f8-a3ef93a779bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6801","nonce":4085056128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6803","nonce":4085056128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6807","nonce":4085056128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":4085056128},{"type":"v1","addr":"172.21.15.196:6805","nonce":4085056128}]},"public_addr":"172.21.15.196:6801/4085056128","cluster_addr":"172.21.15.196:6803/4085056128","heartbeat_back_addr":"172.21.15.196:6807/4085056128","heartbeat_front_addr":"172.21.15.196:6805/4085056128","state":["exists","up"]},{"osd":3,"uuid":"1c13f30d-2850-46f4-a0d5-9f8d96b699ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6811","nonce":3319817620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6813","nonce":3319817620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6817","nonce":3319817620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3319817620},{"type":"v1","addr":"172.21.15.19:6815","nonce":3319817620}]},"public_addr":"172.21.15.19:6811/3319817620","cluster_addr":"172.21.15.19:6813/3319817620","heartbeat_back_addr":"172.21.15.19:6817/3319817620","heartbeat_front_addr":"172.21.15.19:6815/3319817620","state":["exists","up"]},{"osd":4,"uuid":"fa74ad7b-8602-4087-aee4-c6e9632864c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6809","nonce":942057165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6811","nonce":942057165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6815","nonce":942057165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":942057165},{"type":"v1","addr":"172.21.15.111:6813","nonce":942057165}]},"public_addr":"172.21.15.111:6809/942057165","cluster_addr":"172.21.15.111:6811/942057165","heartbeat_back_addr":"172.21.15.111:6815/942057165","heartbeat_front_addr":"172.21.15.111:6813/942057165","state":["exists","up"]},{"osd":5,"uuid":"14c144ac-b587-4412-8aae-b0aef6058124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6809","nonce":919577999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6811","nonce":919577999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6815","nonce":919577999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":919577999},{"type":"v1","addr":"172.21.15.196:6813","nonce":919577999}]},"public_addr":"172.21.15.196:6809/919577999","cluster_addr":"172.21.15.196:6811/919577999","heartbeat_back_addr":"172.21.15.196:6815/919577999","heartbeat_front_addr":"172.21.15.196:6813/919577999","state":["exists","up"]},{"osd":6,"uuid":"531057c9-5241-4e2e-9f6a-5ce00fb4cd26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6819","nonce":3521642658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6821","nonce":3521642658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6825","nonce":3521642658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3521642658},{"type":"v1","addr":"172.21.15.19:6823","nonce":3521642658}]},"public_addr":"172.21.15.19:6819/3521642658","cluster_addr":"172.21.15.19:6821/3521642658","heartbeat_back_addr":"172.21.15.19:6825/3521642658","heartbeat_front_addr":"172.21.15.19:6823/3521642658","state":["exists","up"]},{"osd":7,"uuid":"44f8f0de-ffc4-4b29-b40a-73b485a3c478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6817","nonce":837241728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6819","nonce":837241728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6823","nonce":837241728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837241728},{"type":"v1","addr":"172.21.15.111:6821","nonce":837241728}]},"public_addr":"172.21.15.111:6817/837241728","cluster_addr":"172.21.15.111:6819/837241728","heartbeat_back_addr":"172.21.15.111:6823/837241728","heartbeat_front_addr":"172.21.15.111:6821/837241728","state":["exists","up"]},{"osd":8,"uuid":"8cdff67f-fc25-43ca-99e0-cb2063ee063c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6817","nonce":189709907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6819","nonce":189709907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6823","nonce":189709907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":189709907},{"type":"v1","addr":"172.21.15.196:6821","nonce":189709907}]},"public_addr":"172.21.15.196:6817/189709907","cluster_addr":"172.21.15.196:6819/189709907","heartbeat_back_addr":"172.21.15.196:6823/189709907","heartbeat_front_addr":"172.21.15.196:6821/189709907","state":["exists","up"]},{"osd":9,"uuid":"1f507340-9ceb-4005-9247-bb7fac27bdf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6827","nonce":2115247228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6829","nonce":2115247228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6833","nonce":2115247228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2115247228},{"type":"v1","addr":"172.21.15.19:6831","nonce":2115247228}]},"public_addr":"172.21.15.19:6827/2115247228","cluster_addr":"172.21.15.19:6829/2115247228","heartbeat_back_addr":"172.21.15.19:6833/2115247228","heartbeat_front_addr":"172.21.15.19:6831/2115247228","state":["exists","up"]},{"osd":10,"uuid":"17727a7c-08c8-46ba-b9f9-e0704680e6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6825","nonce":610848665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6827","nonce":610848665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6831","nonce":610848665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":610848665},{"type":"v1","addr":"172.21.15.111:6829","nonce":610848665}]},"public_addr":"172.21.15.111:6825/610848665","cluster_addr":"172.21.15.111:6827/610848665","heartbeat_back_addr":"172.21.15.111:6831/610848665","heartbeat_front_addr":"172.21.15.111:6829/610848665","state":["exists","up"]},{"osd":11,"uuid":"f71692d8-81b5-439b-b1c2-8972209815cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6825","nonce":301665031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6827","nonce":301665031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6831","nonce":301665031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":301665031},{"type":"v1","addr":"172.21.15.196:6829","nonce":301665031}]},"public_addr":"172.21.15.196:6825/301665031","cluster_addr":"172.21.15.196:6827/301665031","heartbeat_back_addr":"172.21.15.196:6831/301665031","heartbeat_front_addr":"172.21.15.196:6829/301665031","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:06.178601+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:30.791308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T03:59:55.613502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:21.427744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:00:47.577876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:12.993161+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:01:38.607152+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:04.929524+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:29.268995+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:02:55.207980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:21.039518+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:03:46.726809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1292798638":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6800/2229567042":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/2828287970":"2024-09-20T03:57:40.326582+0000","172.21.15.19:0/569607576":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/748113766":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3673052449":"2024-09-20T03:57:15.012251+0000","172.21.15.19:0/3967546781":"2024-09-20T03:57:40.326582+0000","172.21.15.19:6801/3897870084":"2024-09-20T03:57:15.012251+0000","172.21.15.19:6800/3897870084":"2024-09-20T03:57:15.012251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:10:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe1400a32c0 msgr2=0x7fe1400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe1400a32c0 0x7fe1400a5780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe144009900 tx=0x7fe1440058d0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541036e0 msgr2=0x7fe154198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541036e0 0x7fe154198b20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00d750 tx=0x7fe13c00dc10 comp rx=0 tx=0).stop 2024-09-19T04:10:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 shutdown_connections 2024-09-19T04:10:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fe1400a32c0 0x7fe1400a5780 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1541082a0 0x7fe15419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541036e0 0x7fe154198b20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 --2- 172.21.15.19:0/570718379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe154102d30 0x7fe1541985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 >> 172.21.15.19:0/570718379 conn(0x7fe1540fc7d0 msgr2=0x7fe154105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 shutdown_connections 2024-09-19T04:10:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:07.553+0000 7fe159b4f640 1 -- 172.21.15.19:0/570718379 wait complete. 2024-09-19T04:10:08.239 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T04:10:08.499 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:08.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:08 smithi019 ceph-mon[30938]: pgmap v456: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T04:10:08.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:08 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/570718379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:10:08.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:08 smithi111 ceph-mon[35365]: pgmap v456: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T04:10:08.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:08 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/570718379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:10:08.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:08 smithi196 ceph-mon[35731]: pgmap v456: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T04:10:08.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:08 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/570718379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T04:10:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 -- 172.21.15.19:0/3309271980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c101420 msgr2=0x7f905c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 --2- 172.21.15.19:0/3309271980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c101420 0x7f905c10d950 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9054009cd0 tx=0x7f905402a8e0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 -- 172.21.15.19:0/3309271980 shutdown_connections 2024-09-19T04:10:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 --2- 172.21.15.19:0/3309271980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c101420 0x7f905c10d950 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 --2- 172.21.15.19:0/3309271980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f905c100a80 0x7f905c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 --2- 172.21.15.19:0/3309271980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f905c106a80 0x7f905c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 -- 172.21.15.19:0/3309271980 >> 172.21.15.19:0/3309271980 conn(0x7f905c0fc7a0 msgr2=0x7f905c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 -- 172.21.15.19:0/3309271980 shutdown_connections 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.004+0000 7f9060958640 1 -- 172.21.15.19:0/3309271980 wait complete. 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9060958640 1 Processor -- start 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9060958640 1 -- start start 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9060958640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f905c100a80 0x7f905c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9060958640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f905c101420 0x7f905c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9060958640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c106a80 0x7f905c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9060958640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f905c110e80 con 0x7f905c106a80 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9060958640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f905c110d00 con 0x7f905c101420 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9060958640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f905c111000 con 0x7f905c100a80 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c106a80 0x7f905c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c106a80 0x7f905c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54664/0 (socket says 172.21.15.19:54664) 2024-09-19T04:10:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905ad9d640 1 -- 172.21.15.19:0/3032417825 learned_addr learned my addr 172.21.15.19:0/3032417825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905a59c640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f905c100a80 0x7f905c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9059d9b640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f905c101420 0x7f905c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905ad9d640 1 -- 172.21.15.19:0/3032417825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f905c100a80 msgr2=0x7f905c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905ad9d640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f905c100a80 0x7f905c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905ad9d640 1 -- 172.21.15.19:0/3032417825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f905c101420 msgr2=0x7f905c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905ad9d640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f905c101420 0x7f905c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f905ad9d640 1 -- 172.21.15.19:0/3032417825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c19d350 con 0x7f905c106a80 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.005+0000 7f9059d9b640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f905c101420 0x7f905c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.006+0000 7f905ad9d640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c106a80 0x7f905c19cc70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9054006010 tx=0x7f9054003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.006+0000 7f904b7fe640 1 -- 172.21.15.19:0/3032417825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054004450 con 0x7f905c106a80 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.006+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c19d5e0 con 0x7f905c106a80 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.006+0000 7f904b7fe640 1 -- 172.21.15.19:0/3032417825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9054003e10 con 0x7f905c106a80 2024-09-19T04:10:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.006+0000 7f904b7fe640 1 -- 172.21.15.19:0/3032417825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540466e0 con 0x7f905c106a80 2024-09-19T04:10:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.006+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c1a4f20 con 0x7f905c106a80 2024-09-19T04:10:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.007+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905c100ee0 con 0x7f905c106a80 2024-09-19T04:10:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.008+0000 7f904b7fe640 1 -- 172.21.15.19:0/3032417825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f90540305a0 con 0x7f905c106a80 2024-09-19T04:10:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.009+0000 7f904b7fe640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90440a3240 0x7f90440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.009+0000 7f905a59c640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90440a3240 0x7f90440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.009+0000 7f904b7fe640 1 -- 172.21.15.19:0/3032417825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9166+0+0 (secure 0 0 0) 0x7f905402e030 con 0x7f905c106a80 2024-09-19T04:10:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.010+0000 7f905a59c640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90440a3240 0x7f90440a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f904c0058c0 tx=0x7f904c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.013+0000 7f904b7fe640 1 -- 172.21.15.19:0/3032417825 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f905404b060 con 0x7f905c106a80 2024-09-19T04:10:09.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.227+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f905c1a5120 con 0x7f905c106a80 2024-09-19T04:10:09.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.229+0000 7f904b7fe640 1 -- 172.21.15.19:0/3032417825 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f90540be1c0 con 0x7f905c106a80 2024-09-19T04:10:09.232 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90440a3240 msgr2=0x7f90440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90440a3240 0x7f90440a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f904c0058c0 tx=0x7f904c00a2d0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c106a80 msgr2=0x7f905c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c106a80 0x7f905c19cc70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9054006010 tx=0x7f9054003c20 comp rx=0 tx=0).stop 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 shutdown_connections 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f90440a3240 0x7f90440a5700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f905c106a80 0x7f905c19cc70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f905c101420 0x7f905c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 --2- 172.21.15.19:0/3032417825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f905c100a80 0x7f905c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:09.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.233+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 >> 172.21.15.19:0/3032417825 conn(0x7f905c0fc7a0 msgr2=0x7f905c104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:09.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.234+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 shutdown_connections 2024-09-19T04:10:09.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:09.234+0000 7f9060958640 1 -- 172.21.15.19:0/3032417825 wait complete. 2024-09-19T04:10:09.559 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a[71574]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.559 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a[71574]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.559 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-mds[71595]: -- 172.21.15.19:0/3366269578 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e01b03f860 con 0x55e01b01c800 2024-09-19T04:10:09.559 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a[71574]: starting mds.a at 2024-09-19T04:10:09.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a[30934]: 2024-09-19T04:10:09.228+0000 7fdba57e7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:10:09.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3032417825' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:10:09.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-mon[30938]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:10:09.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-mon[30938]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:10:09.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/3032417825' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:10:09.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-mon[30938]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:10:09.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:09 smithi019 ceph-mon[30938]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:10:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3032417825' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:10:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mon[35365]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:10:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mon[35365]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:10:09.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/3032417825' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:10:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mon[35365]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:10:09.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mon[35365]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:10:09.656 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h[64991]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.656 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h[64991]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.656 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h[64991]: starting mds.h at 2024-09-19T04:10:09.656 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.657 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.657 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mds[65308]: -- 172.21.15.111:0/395568280 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55587da61860 con 0x55587da3e800 2024-09-19T04:10:09.657 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: starting mds.b at 2024-09-19T04:10:09.657 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k[64384]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.657 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k[64384]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.657 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mds[64388]: -- 172.21.15.111:0/3828549950 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e276ea7860 con 0x55e276e84800 2024-09-19T04:10:09.657 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k[64384]: starting mds.k at 2024-09-19T04:10:09.657 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e[64686]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.658 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e[64686]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.658 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-mds[64690]: -- 172.21.15.111:0/987557325 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b24a41860 con 0x557b24a1f400 2024-09-19T04:10:09.658 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:10:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e[64686]: starting mds.e at 2024-09-19T04:10:09.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3032417825' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T04:10:09.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mon[35731]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T04:10:09.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mon[35731]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:10:09.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/3032417825' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T04:10:09.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mon[35731]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T04:10:09.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mon[35731]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T04:10:09.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l[64882]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l[64882]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mds[64886]: -- 172.21.15.196:0/2049739984 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e402e5860 con 0x564e402c2800 2024-09-19T04:10:09.709 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l[64882]: starting mds.l at 2024-09-19T04:10:09.709 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f[65184]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f[65184]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mds[65188]: -- 172.21.15.196:0/3994636711 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f01e47c20 con 0x556f01e25000 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f[65184]: starting mds.f at 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i[65786]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i[65786]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mds[65790]: -- 172.21.15.196:0/3529019498 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55652b6ad860 con 0x55652b68b400 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i[65786]: starting mds.i at 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c[65484]: ignoring --setuser ceph since I am not root 2024-09-19T04:10:09.710 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c[65484]: ignoring --setgroup ceph since I am not root 2024-09-19T04:10:09.711 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-mds[65488]: -- 172.21.15.196:0/2292192633 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644ab803c20 con 0x5644ab7e1400 2024-09-19T04:10:09.711 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:10:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c[65484]: starting mds.c at 2024-09-19T04:10:09.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:10:10.234 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:10.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:10 smithi019 ceph-mon[30938]: pgmap v457: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T04:10:10.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:10 smithi111 ceph-mon[35365]: pgmap v457: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T04:10:10.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:10 smithi196 ceph-mon[35731]: pgmap v457: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T04:10:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 -- 172.21.15.19:0/3805775276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35641011c0 msgr2=0x7f3564101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 --2- 172.21.15.19:0/3805775276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35641011c0 0x7f3564101620 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3558009900 tx=0x7f355802a410 comp rx=0 tx=0).stop 2024-09-19T04:10:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 -- 172.21.15.19:0/3805775276 shutdown_connections 2024-09-19T04:10:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 --2- 172.21.15.19:0/3805775276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3564101b60 0x7f356410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 --2- 172.21.15.19:0/3805775276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35641011c0 0x7f3564101620 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 --2- 172.21.15.19:0/3805775276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3564105ad0 0x7f3564105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 -- 172.21.15.19:0/3805775276 >> 172.21.15.19:0/3805775276 conn(0x7f3564077210 msgr2=0x7f35640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 -- 172.21.15.19:0/3805775276 shutdown_connections 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.704+0000 7f3569e53640 1 -- 172.21.15.19:0/3805775276 wait complete. 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3569e53640 1 Processor -- start 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3569e53640 1 -- start start 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3569e53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35641011c0 0x7f35641983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3569e53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3564101b60 0x7f35641988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3569e53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3564105ad0 0x7f356419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3569e53640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3564110ef0 con 0x7f3564105ad0 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3569e53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3564110d70 con 0x7f35641011c0 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3569e53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3564111070 con 0x7f3564101b60 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3563fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3564105ad0 0x7f356419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3563fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3564105ad0 0x7f356419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54718/0 (socket says 172.21.15.19:54718) 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f3563fff640 1 -- 172.21.15.19:0/312708486 learned_addr learned my addr 172.21.15.19:0/312708486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.705+0000 7f35637fe640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35641011c0 0x7f35641983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3562ffd640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3564101b60 0x7f35641988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3563fff640 1 -- 172.21.15.19:0/312708486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3564101b60 msgr2=0x7f35641988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3563fff640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3564101b60 0x7f35641988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3563fff640 1 -- 172.21.15.19:0/312708486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35641011c0 msgr2=0x7f35641983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3563fff640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35641011c0 0x7f35641983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3563fff640 1 -- 172.21.15.19:0/312708486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356419d350 con 0x7f3564105ad0 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3563fff640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3564105ad0 0x7f356419cc70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f355c00e9d0 tx=0x7f355c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3560ff9640 1 -- 172.21.15.19:0/312708486 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c00cd80 con 0x7f3564105ad0 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3560ff9640 1 -- 172.21.15.19:0/312708486 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355c005020 con 0x7f3564105ad0 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356419d640 con 0x7f3564105ad0 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3560ff9640 1 -- 172.21.15.19:0/312708486 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0116d0 con 0x7f3564105ad0 2024-09-19T04:10:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.706+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35641a4f20 con 0x7f3564105ad0 2024-09-19T04:10:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.707+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3564101620 con 0x7f3564105ad0 2024-09-19T04:10:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.708+0000 7f3560ff9640 1 -- 172.21.15.19:0/312708486 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f355c0051d0 con 0x7f3564105ad0 2024-09-19T04:10:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.709+0000 7f3560ff9640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f354c0a32c0 0x7f354c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.709+0000 7f35637fe640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f354c0a32c0 0x7f354c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.709+0000 7f3560ff9640 1 -- 172.21.15.19:0/312708486 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9596+0+0 (secure 0 0 0) 0x7f355c0d03c0 con 0x7f3564105ad0 2024-09-19T04:10:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.709+0000 7f35637fe640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f354c0a32c0 0x7f354c0a5780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f355400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.713+0000 7f3560ff9640 1 -- 172.21.15.19:0/312708486 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f355c016030 con 0x7f3564105ad0 2024-09-19T04:10:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:10.927+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f356402d7c0 con 0x7f3564105ad0 2024-09-19T04:10:11.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.269+0000 7f3560ff9640 1 -- 172.21.15.19:0/312708486 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f355c09cb90 con 0x7f3564105ad0 2024-09-19T04:10:11.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f354c0a32c0 msgr2=0x7f354c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:11.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f354c0a32c0 0x7f354c0a5780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f355400b390 comp rx=0 tx=0).stop 2024-09-19T04:10:11.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3564105ad0 msgr2=0x7f356419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:11.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3564105ad0 0x7f356419cc70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f355c00e9d0 tx=0x7f355c00ee90 comp rx=0 tx=0).stop 2024-09-19T04:10:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 shutdown_connections 2024-09-19T04:10:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7f354c0a32c0 0x7f354c0a5780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3564105ad0 0x7f356419cc70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3564101b60 0x7f35641988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 --2- 172.21.15.19:0/312708486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35641011c0 0x7f35641983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 >> 172.21.15.19:0/312708486 conn(0x7f3564077210 msgr2=0x7f35641002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 shutdown_connections 2024-09-19T04:10:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:11.272+0000 7f3569e53640 1 -- 172.21.15.19:0/312708486 wait complete. 2024-09-19T04:10:11.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6832/3809639956,v1:172.21.15.196:6833/3809639956] up:boot 2024-09-19T04:10:11.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6834/3468332580,v1:172.21.15.196:6835/3468332580] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.19:6838/2969382169,v1:172.21.15.19:6840/2969382169] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6836/3634148825,v1:172.21.15.196:6837/3634148825] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6833/1099258553,v1:172.21.15.111:6835/1099258553] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.19:6839/3797545646,v1:172.21.15.19:6841/3797545646] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6832/2369709333,v1:172.21.15.111:6834/2369709333] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.196:6838/48147129,v1:172.21.15.196:6839/48147129] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6836/3049090062,v1:172.21.15.111:6838/3049090062] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: mds.? [v2:172.21.15.111:6837/925633319,v1:172.21.15.111:6839/925633319] up:boot 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:10:11.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:10:11.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:10:11.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:10:11.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:10:11.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:10:11.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:10:11.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:10:11.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:10:11.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:11 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/312708486' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:10:11.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6832/3809639956,v1:172.21.15.196:6833/3809639956] up:boot 2024-09-19T04:10:11.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6834/3468332580,v1:172.21.15.196:6835/3468332580] up:boot 2024-09-19T04:10:11.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.19:6838/2969382169,v1:172.21.15.19:6840/2969382169] up:boot 2024-09-19T04:10:11.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6836/3634148825,v1:172.21.15.196:6837/3634148825] up:boot 2024-09-19T04:10:11.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6833/1099258553,v1:172.21.15.111:6835/1099258553] up:boot 2024-09-19T04:10:11.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.19:6839/3797545646,v1:172.21.15.19:6841/3797545646] up:boot 2024-09-19T04:10:11.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6832/2369709333,v1:172.21.15.111:6834/2369709333] up:boot 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.196:6838/48147129,v1:172.21.15.196:6839/48147129] up:boot 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6836/3049090062,v1:172.21.15.111:6838/3049090062] up:boot 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: mds.? [v2:172.21.15.111:6837/925633319,v1:172.21.15.111:6839/925633319] up:boot 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:10:11.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:10:11.657 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:11 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/312708486' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:10:11.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6832/3809639956,v1:172.21.15.196:6833/3809639956] up:boot 2024-09-19T04:10:11.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6834/3468332580,v1:172.21.15.196:6835/3468332580] up:boot 2024-09-19T04:10:11.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.19:6838/2969382169,v1:172.21.15.19:6840/2969382169] up:boot 2024-09-19T04:10:11.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6836/3634148825,v1:172.21.15.196:6837/3634148825] up:boot 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6833/1099258553,v1:172.21.15.111:6835/1099258553] up:boot 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.19:6839/3797545646,v1:172.21.15.19:6841/3797545646] up:boot 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6832/2369709333,v1:172.21.15.111:6834/2369709333] up:boot 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.196:6838/48147129,v1:172.21.15.196:6839/48147129] up:boot 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6836/3049090062,v1:172.21.15.111:6838/3049090062] up:boot 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: mds.? [v2:172.21.15.111:6837/925633319,v1:172.21.15.111:6839/925633319] up:boot 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T04:10:11.709 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T04:10:11.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T04:10:11.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T04:10:11.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T04:10:11.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='mgr.14150 172.21.15.19:0/3815385090' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T04:10:11.710 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:11 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/312708486' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:10:11.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:10:12.241 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:12.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:12 smithi019 ceph-mon[30938]: pgmap v459: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T04:10:12.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:12 smithi019 ceph-mon[30938]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:10:12.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:12 smithi019 ceph-mon[30938]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:10:12.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:12 smithi019 ceph-mon[30938]: Cluster is now healthy 2024-09-19T04:10:12.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:12 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/312708486' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:10:12.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:12 smithi019 ceph-mon[30938]: fsmap 12 up:standby 2024-09-19T04:10:12.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:12 smithi111 ceph-mon[35365]: pgmap v459: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T04:10:12.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:12 smithi111 ceph-mon[35365]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:10:12.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:12 smithi111 ceph-mon[35365]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:10:12.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:12 smithi111 ceph-mon[35365]: Cluster is now healthy 2024-09-19T04:10:12.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:12 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/312708486' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:10:12.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:12 smithi111 ceph-mon[35365]: fsmap 12 up:standby 2024-09-19T04:10:12.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:12 smithi196 ceph-mon[35731]: pgmap v459: 129 pgs: 129 active+clean; 4.8 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T04:10:12.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:12 smithi196 ceph-mon[35731]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T04:10:12.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:12 smithi196 ceph-mon[35731]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T04:10:12.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:12 smithi196 ceph-mon[35731]: Cluster is now healthy 2024-09-19T04:10:12.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:12 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/312708486' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:10:12.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:12 smithi196 ceph-mon[35731]: fsmap 12 up:standby 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/3871068361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 msgr2=0x7fd4f8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/3871068361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 0x7fd4f8107b90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009930 tx=0x7fd4e002a420 comp rx=0 tx=0).stop 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/3871068361 shutdown_connections 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/3871068361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f81082a0 0x7fd4f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/3871068361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 0x7fd4f8107b90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/3871068361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8102d30 0x7fd4f8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/3871068361 >> 172.21.15.19:0/3871068361 conn(0x7fd4f80fc7d0 msgr2=0x7fd4f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/3871068361 shutdown_connections 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/3871068361 wait complete. 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.710+0000 7fd4fd0d5640 1 Processor -- start 2024-09-19T04:10:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4fd0d5640 1 -- start start 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4fd0d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f8102d30 0x7fd4f8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4fd0d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 0x7fd4f8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4fd0d5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f81082a0 0x7fd4f819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4fd0d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81120a0 con 0x7fd4f81036e0 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4fd0d5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8111f20 con 0x7fd4f8102d30 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4fd0d5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112220 con 0x7fd4f81082a0 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 0x7fd4f8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 0x7fd4f8198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34176/0 (socket says 172.21.15.19:34176) 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f659c640 1 -- 172.21.15.19:0/4001120542 learned_addr learned my addr 172.21.15.19:0/4001120542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f759e640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f81082a0 0x7fd4f819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f6d9d640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f8102d30 0x7fd4f8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f659c640 1 -- 172.21.15.19:0/4001120542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f81082a0 msgr2=0x7fd4f819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f659c640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f81082a0 0x7fd4f819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f659c640 1 -- 172.21.15.19:0/4001120542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f8102d30 msgr2=0x7fd4f8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f659c640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f8102d30 0x7fd4f8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f659c640 1 -- 172.21.15.19:0/4001120542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f819d540 con 0x7fd4f81036e0 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.711+0000 7fd4f759e640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f81082a0 0x7fd4f819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.712+0000 7fd4f659c640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 0x7fd4f8198ad0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0005940 tx=0x7fd4e0031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.712+0000 7fd4effff640 1 -- 172.21.15.19:0/4001120542 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e003f030 con 0x7fd4f81036e0 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.712+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f819d7d0 con 0x7fd4f81036e0 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.712+0000 7fd4effff640 1 -- 172.21.15.19:0/4001120542 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e00043d0 con 0x7fd4f81036e0 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.712+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a5110 con 0x7fd4f81036e0 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.712+0000 7fd4effff640 1 -- 172.21.15.19:0/4001120542 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0049060 con 0x7fd4f81036e0 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.713+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f8103110 con 0x7fd4f81036e0 2024-09-19T04:10:12.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.714+0000 7fd4effff640 1 -- 172.21.15.19:0/4001120542 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd4e0042030 con 0x7fd4f81036e0 2024-09-19T04:10:12.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.715+0000 7fd4effff640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd4e40a3440 0x7fd4e40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:12.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.715+0000 7fd4f6d9d640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd4e40a3440 0x7fd4e40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:12.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.715+0000 7fd4effff640 1 -- 172.21.15.19:0/4001120542 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9596+0+0 (secure 0 0 0) 0x7fd4e003b7f0 con 0x7fd4f81036e0 2024-09-19T04:10:12.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.716+0000 7fd4f6d9d640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd4e40a3440 0x7fd4e40a5900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8005fd0 tx=0x7fd4e8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:12.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.719+0000 7fd4effff640 1 -- 172.21.15.19:0/4001120542 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4e00bc800 con 0x7fd4f81036e0 2024-09-19T04:10:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:12.896+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd4f810a2b0 con 0x7fd4f81036e0 2024-09-19T04:10:13.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.278+0000 7fd4effff640 1 -- 172.21.15.19:0/4001120542 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7fd4e00bc100 con 0x7fd4f81036e0 2024-09-19T04:10:13.281 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:10:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.281+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd4e40a3440 msgr2=0x7fd4e40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.281+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd4e40a3440 0x7fd4e40a5900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8005fd0 tx=0x7fd4e8009500 comp rx=0 tx=0).stop 2024-09-19T04:10:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.281+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 msgr2=0x7fd4f8198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.281+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 0x7fd4f8198ad0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0005940 tx=0x7fd4e0031b50 comp rx=0 tx=0).stop 2024-09-19T04:10:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.282+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 shutdown_connections 2024-09-19T04:10:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.282+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd4e40a3440 0x7fd4e40a5900 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.282+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f81082a0 0x7fd4f819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.282+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f81036e0 0x7fd4f8198ad0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.282+0000 7fd4fd0d5640 1 --2- 172.21.15.19:0/4001120542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f8102d30 0x7fd4f8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.282+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 >> 172.21.15.19:0/4001120542 conn(0x7fd4f80fc7d0 msgr2=0x7fd4f8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.282+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 shutdown_connections 2024-09-19T04:10:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:13.282+0000 7fd4fd0d5640 1 -- 172.21.15.19:0/4001120542 wait complete. 2024-09-19T04:10:13.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:13 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4001120542' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:13.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:13 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/4001120542' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:13.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:13 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/4001120542' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:14.038 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:10:14.302 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:14.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:14 smithi019 ceph-mon[30938]: pgmap v460: 129 pgs: 129 active+clean; 4.8 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T04:10:14.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:14 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/4001120542' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:14.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:14 smithi019 ceph-mon[30938]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:10:14.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:14 smithi111 ceph-mon[35365]: pgmap v460: 129 pgs: 129 active+clean; 4.8 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T04:10:14.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:14 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/4001120542' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:14.656 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:14 smithi111 ceph-mon[35365]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:10:14.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:14 smithi196 ceph-mon[35731]: pgmap v460: 129 pgs: 129 active+clean; 4.8 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T04:10:14.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:14 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/4001120542' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:14.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:14 smithi196 ceph-mon[35731]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 -- 172.21.15.19:0/401629796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 msgr2=0x7fa34810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 --2- 172.21.15.19:0/401629796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 0x7fa34810bb50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009900 tx=0x7fa33c02a480 comp rx=0 tx=0).stop 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 -- 172.21.15.19:0/401629796 shutdown_connections 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 --2- 172.21.15.19:0/401629796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34810c170 0x7fa34810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 --2- 172.21.15.19:0/401629796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 0x7fa34810bb50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 --2- 172.21.15.19:0/401629796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348102d40 0x7fa348103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 -- 172.21.15.19:0/401629796 >> 172.21.15.19:0/401629796 conn(0x7fa3480fc7a0 msgr2=0x7fa3480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 -- 172.21.15.19:0/401629796 shutdown_connections 2024-09-19T04:10:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.748+0000 7fa34d451640 1 -- 172.21.15.19:0/401629796 wait complete. 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa34d451640 1 Processor -- start 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa34d451640 1 -- start start 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa34d451640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348102d40 0x7fa3481983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa34d451640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 0x7fa348198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa34d451640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34810c170 0x7fa34819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa34d451640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa348111f70 con 0x7fa348103660 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa34d451640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa348111df0 con 0x7fa34810c170 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa34d451640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3481120f0 con 0x7fa348102d40 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa3467fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 0x7fa348198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa3467fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 0x7fa348198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34194/0 (socket says 172.21.15.19:34194) 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa3467fc640 1 -- 172.21.15.19:0/1308370664 learned_addr learned my addr 172.21.15.19:0/1308370664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa3477fe640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34810c170 0x7fa34819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.749+0000 7fa346ffd640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348102d40 0x7fa3481983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa3467fc640 1 -- 172.21.15.19:0/1308370664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348102d40 msgr2=0x7fa3481983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa3467fc640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348102d40 0x7fa3481983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa3467fc640 1 -- 172.21.15.19:0/1308370664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34810c170 msgr2=0x7fa34819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa3467fc640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34810c170 0x7fa34819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa3467fc640 1 -- 172.21.15.19:0/1308370664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34819d380 con 0x7fa348103660 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa346ffd640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348102d40 0x7fa3481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa3467fc640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 0x7fa348198910 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa33c006010 tx=0x7fa33c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa32ffff640 1 -- 172.21.15.19:0/1308370664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c03f030 con 0x7fa348103660 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34819d610 con 0x7fa348103660 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa32ffff640 1 -- 172.21.15.19:0/1308370664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33c004440 con 0x7fa348103660 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa32ffff640 1 -- 172.21.15.19:0/1308370664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c035070 con 0x7fa348103660 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.750+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3481a4f50 con 0x7fa348103660 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.751+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34806b070 con 0x7fa348103660 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.752+0000 7fa32ffff640 1 -- 172.21.15.19:0/1308370664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa33c042030 con 0x7fa348103660 2024-09-19T04:10:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.753+0000 7fa32ffff640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa3300a3240 0x7fa3300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.753+0000 7fa346ffd640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa3300a3240 0x7fa3300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.753+0000 7fa32ffff640 1 -- 172.21.15.19:0/1308370664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9063+0+0 (secure 0 0 0) 0x7fa33c0f0050 con 0x7fa348103660 2024-09-19T04:10:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.754+0000 7fa346ffd640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa3300a3240 0x7fa3300a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa33800abe0 tx=0x7fa33800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.757+0000 7fa32ffff640 1 -- 172.21.15.19:0/1308370664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa33c0bca30 con 0x7fa348103660 2024-09-19T04:10:14.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:14.934+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa3481071e0 con 0x7fa348103660 2024-09-19T04:10:15.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.285+0000 7fa32ffff640 1 -- 172.21.15.19:0/1308370664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7fa33c0bc330 con 0x7fa348103660 2024-09-19T04:10:15.288 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa3300a3240 msgr2=0x7fa3300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa3300a3240 0x7fa3300a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa33800abe0 tx=0x7fa33800a5c0 comp rx=0 tx=0).stop 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 msgr2=0x7fa348198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 0x7fa348198910 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa33c006010 tx=0x7fa33c004040 comp rx=0 tx=0).stop 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 shutdown_connections 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fa3300a3240 0x7fa3300a5700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34810c170 0x7fa34819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348103660 0x7fa348198910 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 --2- 172.21.15.19:0/1308370664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348102d40 0x7fa3481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 >> 172.21.15.19:0/1308370664 conn(0x7fa3480fc7a0 msgr2=0x7fa3480fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:15.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 shutdown_connections 2024-09-19T04:10:15.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:15.289+0000 7fa34d451640 1 -- 172.21.15.19:0/1308370664 wait complete. 2024-09-19T04:10:15.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:15 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1308370664' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:15.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:15 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1308370664' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:15.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:15 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1308370664' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:16.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 0518de98-763b-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-19T04:10:16.399 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:10:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:16 smithi019 ceph-mon[30938]: pgmap v462: 97 pgs: 97 active+clean; 577 KiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:10:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:16 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1308370664' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:16.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:16 smithi019 ceph-mon[30938]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:10:16.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:16 smithi111 ceph-mon[35365]: pgmap v462: 97 pgs: 97 active+clean; 577 KiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:10:16.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:16 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1308370664' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:16.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:16 smithi111 ceph-mon[35365]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:16 smithi196 ceph-mon[35731]: pgmap v462: 97 pgs: 97 active+clean; 577 KiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:16 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1308370664' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:16.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:16 smithi196 ceph-mon[35731]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 -- 172.21.15.19:0/2626352139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 msgr2=0x7fd81c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 --2- 172.21.15.19:0/2626352139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 0x7fd81c10ea90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ca0 tx=0x7fd81402a8c0 comp rx=0 tx=0).stop 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 -- 172.21.15.19:0/2626352139 shutdown_connections 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 --2- 172.21.15.19:0/2626352139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 0x7fd81c10ea90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 --2- 172.21.15.19:0/2626352139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c102790 0x7fd81c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 --2- 172.21.15.19:0/2626352139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c101de0 0x7fd81c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 -- 172.21.15.19:0/2626352139 >> 172.21.15.19:0/2626352139 conn(0x7fd81c0fd8c0 msgr2=0x7fd81c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 -- 172.21.15.19:0/2626352139 shutdown_connections 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.864+0000 7fd820a31640 1 -- 172.21.15.19:0/2626352139 wait complete. 2024-09-19T04:10:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd820a31640 1 Processor -- start 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd820a31640 1 -- start start 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd820a31640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c101de0 0x7fd81c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd820a31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c102790 0x7fd81c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd820a31640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 0x7fd81c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd820a31640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c111f70 con 0x7fd81c106f10 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd820a31640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c111df0 con 0x7fd81c101de0 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd820a31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1120f0 con 0x7fd81c102790 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 0x7fd81c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 0x7fd81c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34220/0 (socket says 172.21.15.19:34220) 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81ad9d640 1 -- 172.21.15.19:0/1831008685 learned_addr learned my addr 172.21.15.19:0/1831008685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81a59c640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c101de0 0x7fd81c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd811d9b640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c102790 0x7fd81c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81ad9d640 1 -- 172.21.15.19:0/1831008685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c102790 msgr2=0x7fd81c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81ad9d640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c102790 0x7fd81c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81ad9d640 1 -- 172.21.15.19:0/1831008685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c101de0 msgr2=0x7fd81c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81ad9d640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c101de0 0x7fd81c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd81ad9d640 1 -- 172.21.15.19:0/1831008685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c1a6780 con 0x7fd81c106f10 2024-09-19T04:10:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.865+0000 7fd811d9b640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c102790 0x7fd81c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:10:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.866+0000 7fd81ad9d640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 0x7fd81c074f70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd814031a60 tx=0x7fd814031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:16.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.866+0000 7fd813fff640 1 -- 172.21.15.19:0/1831008685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814003c30 con 0x7fd81c106f10 2024-09-19T04:10:16.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.866+0000 7fd813fff640 1 -- 172.21.15.19:0/1831008685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd814003dd0 con 0x7fd81c106f10 2024-09-19T04:10:16.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.866+0000 7fd813fff640 1 -- 172.21.15.19:0/1831008685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8140047e0 con 0x7fd81c106f10 2024-09-19T04:10:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.866+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c1a6a10 con 0x7fd81c106f10 2024-09-19T04:10:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.866+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1a6f50 con 0x7fd81c106f10 2024-09-19T04:10:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.867+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81c1021c0 con 0x7fd81c106f10 2024-09-19T04:10:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.868+0000 7fd813fff640 1 -- 172.21.15.19:0/1831008685 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd81402f790 con 0x7fd81c106f10 2024-09-19T04:10:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.869+0000 7fd813fff640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8080a3240 0x7fd8080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:10:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.869+0000 7fd81a59c640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8080a3240 0x7fd8080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:10:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.869+0000 7fd813fff640 1 -- 172.21.15.19:0/1831008685 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8496+0+0 (secure 0 0 0) 0x7fd8140f0f30 con 0x7fd81c106f10 2024-09-19T04:10:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.870+0000 7fd81a59c640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8080a3240 0x7fd8080a5700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005e10 tx=0x7fd80c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:10:16.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:16.873+0000 7fd813fff640 1 -- 172.21.15.19:0/1831008685 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd8140bdb10 con 0x7fd81c106f10 2024-09-19T04:10:17.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.049+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd81c1a71b0 con 0x7fd81c106f10 2024-09-19T04:10:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.292+0000 7fd813fff640 1 -- 172.21.15.19:0/1831008685 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) ==== 178+0+0 (secure 0 0 0) 0x7fd8140bd410 con 0x7fd81c106f10 2024-09-19T04:10:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data_ec' removed 2024-09-19T04:10:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.295+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8080a3240 msgr2=0x7fd8080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.295+0000 7fd820a31640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8080a3240 0x7fd8080a5700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005e10 tx=0x7fd80c005da0 comp rx=0 tx=0).stop 2024-09-19T04:10:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.295+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 msgr2=0x7fd81c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:10:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.295+0000 7fd820a31640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 0x7fd81c074f70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd814031a60 tx=0x7fd814031a90 comp rx=0 tx=0).stop 2024-09-19T04:10:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.296+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 shutdown_connections 2024-09-19T04:10:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.296+0000 7fd820a31640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:6800/2387193681,v1:172.21.15.19:6801/2387193681] conn(0x7fd8080a3240 0x7fd8080a5700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.296+0000 7fd820a31640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd81c106f10 0x7fd81c074f70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.296+0000 7fd820a31640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c102790 0x7fd81c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.296+0000 7fd820a31640 1 --2- 172.21.15.19:0/1831008685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c101de0 0x7fd81c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:10:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.296+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 >> 172.21.15.19:0/1831008685 conn(0x7fd81c0fd8c0 msgr2=0x7fd81c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:10:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.296+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 shutdown_connections 2024-09-19T04:10:17.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-19T04:10:17.296+0000 7fd820a31640 1 -- 172.21.15.19:0/1831008685 wait complete. 2024-09-19T04:10:17.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:17 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1831008685' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:17.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:17 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1831008685' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:17.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:17 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1831008685' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T04:10:18.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:10:18.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:10:18.118 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:10:18.151 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T04:10:18.152 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:10:18.169 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:10:18.186 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:10:18.210 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T04:10:18.210 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T04:10:18.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:10:18.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 systemd[1]: Stopping Ceph mon.a for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:18.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:18 smithi111 ceph-mon[35365]: pgmap v464: 33 pgs: 33 active+clean; 577 KiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:10:18.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:18 smithi111 ceph-mon[35365]: from='client.? 172.21.15.19:0/1831008685' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:18.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:18 smithi111 ceph-mon[35365]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:10:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:18 smithi196 ceph-mon[35731]: pgmap v464: 33 pgs: 33 active+clean; 577 KiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:10:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:18 smithi196 ceph-mon[35731]: from='client.? 172.21.15.19:0/1831008685' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:18.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:18 smithi196 ceph-mon[35731]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:10:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 ceph-mon[30938]: pgmap v464: 33 pgs: 33 active+clean; 577 KiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T04:10:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 ceph-mon[30938]: from='client.? 172.21.15.19:0/1831008685' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T04:10:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 ceph-mon[30938]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T04:10:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a[30934]: 2024-09-19T04:10:18.537+0000 7fdbaaff2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a[30934]: 2024-09-19T04:10:18.537+0000 7fdbaaff2640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T04:10:18.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 podman[91786]: 2024-09-19 04:10:18.570398326 +0000 UTC m=+0.152306281 container died 4593a6e9f16a8508881d3133c95900c31aa771ab110d511be740d706420259e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T04:10:19.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 podman[91786]: 2024-09-19 04:10:18.96412258 +0000 UTC m=+0.546030526 container cleanup 4593a6e9f16a8508881d3133c95900c31aa771ab110d511be740d706420259e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T04:10:19.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:18 smithi019 bash[91786]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:10:19.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:19 smithi019 podman[91802]: 2024-09-19 04:10:19.340986233 +0000 UTC m=+0.766716122 container remove 4593a6e9f16a8508881d3133c95900c31aa771ab110d511be740d706420259e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:10:19.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:19 smithi019 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T04:10:19.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:19 smithi019 systemd[1]: Stopped Ceph mon.a for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:10:19.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 19 04:10:19 smithi019 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a.service: Consumed 15.361s CPU time. 2024-09-19T04:10:19.550 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T04:10:19.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:19.593 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T04:10:19.593 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T04:10:19.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.b 2024-09-19T04:10:19.905 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:19 smithi111 systemd[1]: Stopping Ceph mon.b for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:20.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-b[35361]: 2024-09-19T04:10:19.925+0000 7f54734f8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:20.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-b[35361]: 2024-09-19T04:10:19.925+0000 7f54734f8640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T04:10:20.405 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:20 smithi111 podman[68191]: 2024-09-19 04:10:20.171159131 +0000 UTC m=+0.382163382 container died 5e9a143ee94e4094e1c26589222c32208c5aa39b071b3b914b018fd3a53519e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T04:10:20.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:20 smithi111 podman[68191]: 2024-09-19 04:10:20.605372508 +0000 UTC m=+0.816376757 container cleanup 5e9a143ee94e4094e1c26589222c32208c5aa39b071b3b914b018fd3a53519e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T04:10:20.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:20 smithi111 bash[68191]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-b 2024-09-19T04:10:21.038 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T04:10:21.060 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:20 smithi111 podman[68203]: 2024-09-19 04:10:20.798673017 +0000 UTC m=+0.623914915 container remove 5e9a143ee94e4094e1c26589222c32208c5aa39b071b3b914b018fd3a53519e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T04:10:21.060 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:21 smithi111 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-19T04:10:21.060 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:21 smithi111 systemd[1]: Stopped Ceph mon.b for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:10:21.061 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 19 04:10:21 smithi111 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.b.service: Consumed 8.057s CPU time. 2024-09-19T04:10:21.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:21.094 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T04:10:21.095 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T04:10:21.095 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.c 2024-09-19T04:10:21.421 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:21 smithi196 systemd[1]: Stopping Ceph mon.c for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:21.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:21 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-c[35716]: 2024-09-19T04:10:21.419+0000 7fa9becc6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:21.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:21 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-c[35716]: 2024-09-19T04:10:21.419+0000 7fa9becc6640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T04:10:21.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:21 smithi196 podman[68716]: 2024-09-19 04:10:21.481226924 +0000 UTC m=+0.192495039 container died 9a1e0c7faa3bdec14ff6b5e410ab1dae5df2df3aa19fba2f16ab15cc38ef70e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T04:10:22.148 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:21 smithi196 podman[68716]: 2024-09-19 04:10:21.887394845 +0000 UTC m=+0.598662960 container cleanup 9a1e0c7faa3bdec14ff6b5e410ab1dae5df2df3aa19fba2f16ab15cc38ef70e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T04:10:22.148 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 04:10:21 smithi196 bash[68716]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mon-c 2024-09-19T04:10:22.307 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T04:10:22.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:22.357 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T04:10:22.357 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T04:10:22.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:10:22.679 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:22 smithi019 systemd[1]: Stopping Ceph mgr.x for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:22.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:22 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x[31164]: 2024-09-19T04:10:22.677+0000 7f0745975640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:22.931 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:22 smithi019 podman[91908]: 2024-09-19 04:10:22.708569627 +0000 UTC m=+0.153971704 container died ffebeb9c157efaf2b4c8966bb0752a35dbe8e9dcd503975cdc838f92f57d806f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T04:10:23.431 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:23 smithi019 podman[91908]: 2024-09-19 04:10:23.22841073 +0000 UTC m=+0.673812796 container cleanup ffebeb9c157efaf2b4c8966bb0752a35dbe8e9dcd503975cdc838f92f57d806f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T04:10:23.431 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:23 smithi019 bash[91908]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x 2024-09-19T04:10:23.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T04:10:23.742 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:23 smithi019 podman[91920]: 2024-09-19 04:10:23.480075174 +0000 UTC m=+0.768504925 container remove ffebeb9c157efaf2b4c8966bb0752a35dbe8e9dcd503975cdc838f92f57d806f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:10:23.742 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:23 smithi019 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-19T04:10:23.742 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:23 smithi019 systemd[1]: Stopped Ceph mgr.x for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:10:23.742 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 19 04:10:23 smithi019 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.x.service: Consumed 42.660s CPU time. 2024-09-19T04:10:23.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:23.757 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T04:10:23.757 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T04:10:23.757 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.y 2024-09-19T04:10:24.107 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 04:10:23 smithi111 systemd[1]: Stopping Ceph mgr.y for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:24.405 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 04:10:24 smithi111 podman[68322]: 2024-09-19 04:10:24.106026955 +0000 UTC m=+0.145882580 container died 539ccb18f7b0082ff31a9965c240f9b9f05adda35f79dfc5038524c546d639ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T04:10:24.807 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 04:10:24 smithi111 podman[68322]: 2024-09-19 04:10:24.538230741 +0000 UTC m=+0.578086368 container cleanup 539ccb18f7b0082ff31a9965c240f9b9f05adda35f79dfc5038524c546d639ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T04:10:24.807 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 04:10:24 smithi111 bash[68322]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y 2024-09-19T04:10:25.096 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 04:10:24 smithi111 podman[68335]: 2024-09-19 04:10:24.806583852 +0000 UTC m=+0.692006477 container remove 539ccb18f7b0082ff31a9965c240f9b9f05adda35f79dfc5038524c546d639ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T04:10:25.096 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 19 04:10:24 smithi111 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-19T04:10:25.114 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T04:10:25.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:25.152 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T04:10:25.152 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T04:10:25.152 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.z 2024-09-19T04:10:25.458 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 04:10:25 smithi196 systemd[1]: Stopping Ceph mgr.z for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:25.723 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 04:10:25 smithi196 podman[68838]: 2024-09-19 04:10:25.496020233 +0000 UTC m=+0.138013750 container died 1dfea3d0731125a8246c6c0e5e3f3e8f61f6f82181a4a57b3600ed58b693d834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T04:10:26.179 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 04:10:25 smithi196 podman[68838]: 2024-09-19 04:10:25.893119875 +0000 UTC m=+0.535113390 container cleanup 1dfea3d0731125a8246c6c0e5e3f3e8f61f6f82181a4a57b3600ed58b693d834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T04:10:26.179 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 04:10:25 smithi196 bash[68838]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mgr-z 2024-09-19T04:10:26.330 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T04:10:26.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:26.367 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T04:10:26.367 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T04:10:26.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.0 2024-09-19T04:10:26.681 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:26 smithi019 systemd[1]: Stopping Ceph osd.0 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:27.182 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:26 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0[44515]: 2024-09-19T04:10:26.682+0000 7f735dddf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:27.182 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:26 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0[44515]: 2024-09-19T04:10:26.682+0000 7f735dddf640 -1 osd.0 97 *** Got signal Terminated *** 2024-09-19T04:10:27.182 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:26 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0[44515]: 2024-09-19T04:10:26.682+0000 7f735dddf640 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:10:32.431 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:32 smithi019 podman[92030]: 2024-09-19 04:10:32.125092764 +0000 UTC m=+5.556531834 container died 93ade25bb40e7a8b0213a60fa8cc63264510f3da1fac439b1773d11c028c349b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T04:10:32.827 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:32 smithi019 podman[92030]: 2024-09-19 04:10:32.641438492 +0000 UTC m=+6.072877563 container cleanup 93ade25bb40e7a8b0213a60fa8cc63264510f3da1fac439b1773d11c028c349b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:10:32.827 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:32 smithi019 bash[92030]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0 2024-09-19T04:10:33.181 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:32 smithi019 podman[92045]: 2024-09-19 04:10:32.826243839 +0000 UTC m=+0.693265791 container remove 93ade25bb40e7a8b0213a60fa8cc63264510f3da1fac439b1773d11c028c349b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:10:33.522 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:33 smithi019 podman[92137]: 2024-09-19 04:10:33.161455999 +0000 UTC m=+0.016585708 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:10:33.523 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:33 smithi019 podman[92137]: 2024-09-19 04:10:33.269344136 +0000 UTC m=+0.124473887 container create 2b68791b8830bcfc09a8bef719aba7abe88a381b3a41044d06cba714f95aa7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:10:33.931 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:33 smithi019 podman[92137]: 2024-09-19 04:10:33.521699543 +0000 UTC m=+0.376829253 container init 2b68791b8830bcfc09a8bef719aba7abe88a381b3a41044d06cba714f95aa7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:10:33.931 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:33 smithi019 podman[92137]: 2024-09-19 04:10:33.525383566 +0000 UTC m=+0.380513278 container start 2b68791b8830bcfc09a8bef719aba7abe88a381b3a41044d06cba714f95aa7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:10:33.932 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:33 smithi019 podman[92137]: 2024-09-19 04:10:33.630129385 +0000 UTC m=+0.485259106 container attach 2b68791b8830bcfc09a8bef719aba7abe88a381b3a41044d06cba714f95aa7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:10:33.932 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 19 04:10:33 smithi019 podman[92137]: 2024-09-19 04:10:33.718299528 +0000 UTC m=+0.573429237 container died 2b68791b8830bcfc09a8bef719aba7abe88a381b3a41044d06cba714f95aa7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T04:10:34.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T04:10:34.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:34.468 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T04:10:34.468 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T04:10:34.468 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.1 2024-09-19T04:10:34.785 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:34 smithi111 systemd[1]: Stopping Ceph osd.1 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:35.155 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:34 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1[39818]: 2024-09-19T04:10:34.784+0000 7fb13408b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:35.155 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:34 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1[39818]: 2024-09-19T04:10:34.784+0000 7fb13408b640 -1 osd.1 97 *** Got signal Terminated *** 2024-09-19T04:10:35.155 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:34 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1[39818]: 2024-09-19T04:10:34.784+0000 7fb13408b640 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:10:40.405 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:40 smithi111 podman[68448]: 2024-09-19 04:10:40.142115656 +0000 UTC m=+5.478630475 container died b8c7f6dce8745401769369b964411821409f8981f8d42911a17ce0b6923acef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T04:10:40.803 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:40 smithi111 podman[68448]: 2024-09-19 04:10:40.551099913 +0000 UTC m=+5.887614734 container cleanup b8c7f6dce8745401769369b964411821409f8981f8d42911a17ce0b6923acef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T04:10:40.803 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:40 smithi111 bash[68448]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1 2024-09-19T04:10:41.155 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:40 smithi111 podman[68464]: 2024-09-19 04:10:40.80290222 +0000 UTC m=+0.659189284 container remove b8c7f6dce8745401769369b964411821409f8981f8d42911a17ce0b6923acef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T04:10:41.473 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:41 smithi111 podman[68551]: 2024-09-19 04:10:41.069856257 +0000 UTC m=+0.015924320 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:10:41.474 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:41 smithi111 podman[68551]: 2024-09-19 04:10:41.195529856 +0000 UTC m=+0.141597946 container create bdd91eaad248310fa702cf36695c000fe035c6d7825937578bd7f458ae9017e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T04:10:41.905 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:41 smithi111 podman[68551]: 2024-09-19 04:10:41.473079694 +0000 UTC m=+0.419147740 container init bdd91eaad248310fa702cf36695c000fe035c6d7825937578bd7f458ae9017e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:10:41.905 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:41 smithi111 podman[68551]: 2024-09-19 04:10:41.477408601 +0000 UTC m=+0.423476647 container start bdd91eaad248310fa702cf36695c000fe035c6d7825937578bd7f458ae9017e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:10:41.905 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:41 smithi111 podman[68551]: 2024-09-19 04:10:41.632196875 +0000 UTC m=+0.578264929 container attach bdd91eaad248310fa702cf36695c000fe035c6d7825937578bd7f458ae9017e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T04:10:41.905 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 19 04:10:41 smithi111 podman[68551]: 2024-09-19 04:10:41.676134895 +0000 UTC m=+0.622202941 container died bdd91eaad248310fa702cf36695c000fe035c6d7825937578bd7f458ae9017e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T04:10:42.356 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T04:10:42.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:42.398 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T04:10:42.398 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T04:10:42.399 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.2 2024-09-19T04:10:42.706 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:42 smithi196 systemd[1]: Stopping Ceph osd.2 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:42.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2[40533]: 2024-09-19T04:10:42.703+0000 7f4142237640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:42.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2[40533]: 2024-09-19T04:10:42.703+0000 7f4142237640 -1 osd.2 97 *** Got signal Terminated *** 2024-09-19T04:10:42.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2[40533]: 2024-09-19T04:10:42.703+0000 7f4142237640 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:10:48.365 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:48 smithi196 podman[68971]: 2024-09-19 04:10:48.047209319 +0000 UTC m=+5.446576417 container died 9ffcb7f3c614c7fb4256b9e754c20706233e265fdf0331e2a8622eb9de8a1f17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T04:10:48.699 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:48 smithi196 podman[68971]: 2024-09-19 04:10:48.463409862 +0000 UTC m=+5.862776963 container cleanup 9ffcb7f3c614c7fb4256b9e754c20706233e265fdf0331e2a8622eb9de8a1f17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T04:10:48.699 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:48 smithi196 bash[68971]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2 2024-09-19T04:10:48.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:48 smithi196 podman[68986]: 2024-09-19 04:10:48.698272894 +0000 UTC m=+0.647507121 container remove 9ffcb7f3c614c7fb4256b9e754c20706233e265fdf0331e2a8622eb9de8a1f17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:10:49.458 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:49 smithi196 podman[69075]: 2024-09-19 04:10:49.023302035 +0000 UTC m=+0.016142802 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:10:49.459 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:49 smithi196 podman[69075]: 2024-09-19 04:10:49.157494843 +0000 UTC m=+0.150335652 container create 845e2d3b496cb1ce849b292e9a1b8f3be623db9be251dbd9af77e25dd7bbbf18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T04:10:49.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:49 smithi196 podman[69075]: 2024-09-19 04:10:49.501708654 +0000 UTC m=+0.494549421 container init 845e2d3b496cb1ce849b292e9a1b8f3be623db9be251dbd9af77e25dd7bbbf18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:10:49.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:49 smithi196 podman[69075]: 2024-09-19 04:10:49.505015857 +0000 UTC m=+0.497856625 container start 845e2d3b496cb1ce849b292e9a1b8f3be623db9be251dbd9af77e25dd7bbbf18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T04:10:49.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:49 smithi196 podman[69075]: 2024-09-19 04:10:49.61018256 +0000 UTC m=+0.603023328 container attach 845e2d3b496cb1ce849b292e9a1b8f3be623db9be251dbd9af77e25dd7bbbf18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T04:10:49.959 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 04:10:49 smithi196 podman[69075]: 2024-09-19 04:10:49.697411949 +0000 UTC m=+0.690252722 container died 845e2d3b496cb1ce849b292e9a1b8f3be623db9be251dbd9af77e25dd7bbbf18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T04:10:50.390 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T04:10:50.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:50.433 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T04:10:50.433 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T04:10:50.433 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.3 2024-09-19T04:10:50.754 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:50 smithi019 systemd[1]: Stopping Ceph osd.3 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:51.026 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:50 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3[51401]: 2024-09-19T04:10:50.752+0000 7f3be0a30640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:51.026 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:50 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3[51401]: 2024-09-19T04:10:50.752+0000 7f3be0a30640 -1 osd.3 97 *** Got signal Terminated *** 2024-09-19T04:10:51.026 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:50 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3[51401]: 2024-09-19T04:10:50.752+0000 7f3be0a30640 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:10:53.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:10:53 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:10:53.202+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:10:54.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:10:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:10:54.177+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:10:55.181 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:54 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3[51401]: 2024-09-19T04:10:54.918+0000 7f3bdd046640 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.740499+0000 front 2024-09-19T04:10:28.740575+0000 (oldest deadline 2024-09-19T04:10:54.640704+0000) 2024-09-19T04:10:55.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:10:55 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:10:55.138+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:10:55.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:10:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:10:55.132+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:10:56.135 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:10:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:10:55.735+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:10:56.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:10:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:10:56.134+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:10:56.564 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:56 smithi019 podman[92241]: 2024-09-19 04:10:56.253682336 +0000 UTC m=+5.615090985 container died 41894d07bdfec90ef808f6e379fcb413566a0c3378c12eb39936c810b1fcc1de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:10:56.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:10:56 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:10:56.169+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:10:56.931 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:56 smithi019 podman[92241]: 2024-09-19 04:10:56.761804519 +0000 UTC m=+6.123213150 container cleanup 41894d07bdfec90ef808f6e379fcb413566a0c3378c12eb39936c810b1fcc1de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:10:56.931 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:56 smithi019 bash[92241]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3 2024-09-19T04:10:56.958 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:10:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:10:56.691+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:10:57.309 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:56 smithi019 podman[92271]: 2024-09-19 04:10:56.980111287 +0000 UTC m=+0.718185244 container remove 41894d07bdfec90ef808f6e379fcb413566a0c3378c12eb39936c810b1fcc1de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T04:10:57.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:10:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:10:57.139+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:10:57.597 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:57 smithi019 podman[92358]: 2024-09-19 04:10:57.211940707 +0000 UTC m=+0.016377485 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:10:57.597 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:57 smithi019 podman[92358]: 2024-09-19 04:10:57.472719558 +0000 UTC m=+0.277156365 container create b4799efd81eda8d8564e2e6d0041c89735621b772a502d295d9aad5be346fb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T04:10:57.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:10:57 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:10:57.182+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:10:57.927 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:10:57 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:10:57.770+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:10:57.927 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:57 smithi019 podman[92358]: 2024-09-19 04:10:57.725187531 +0000 UTC m=+0.529624302 container init b4799efd81eda8d8564e2e6d0041c89735621b772a502d295d9aad5be346fb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T04:10:57.927 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:57 smithi019 podman[92358]: 2024-09-19 04:10:57.728754899 +0000 UTC m=+0.533191669 container start b4799efd81eda8d8564e2e6d0041c89735621b772a502d295d9aad5be346fb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T04:10:57.927 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:57 smithi019 podman[92358]: 2024-09-19 04:10:57.808438394 +0000 UTC m=+0.612875156 container attach b4799efd81eda8d8564e2e6d0041c89735621b772a502d295d9aad5be346fb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:10:57.927 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 19 04:10:57 smithi019 podman[92358]: 2024-09-19 04:10:57.92625581 +0000 UTC m=+0.730692575 container died b4799efd81eda8d8564e2e6d0041c89735621b772a502d295d9aad5be346fb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:10:58.129 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:10:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:10:57.740+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:10:58.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:10:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:10:58.128+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:10:58.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:10:58 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:10:58.168+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:10:58.715 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T04:10:58.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:10:58.758 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T04:10:58.759 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T04:10:58.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.4 2024-09-19T04:10:59.081 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:10:58 smithi111 systemd[1]: Stopping Ceph osd.4 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:10:59.083 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:10:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:10:58.762+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:10:59.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:10:58 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:10:58.754+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:10:59.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:10:59 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:10:59.192+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:10:59.405 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:10:59 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:10:59.080+0000 7fa74ab06640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:10:59.405 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:10:59 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:10:59.080+0000 7fa74ab06640 -1 osd.4 97 *** Got signal Terminated *** 2024-09-19T04:10:59.405 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:10:59 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:10:59.080+0000 7fa74ab06640 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:10:59.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:10:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:10:59.081+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:00.075 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:10:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:10:59.787+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:00.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:10:59 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:10:59.729+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:00.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:00.074+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:00.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:00 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:00.161+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:01.071 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:00.777+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:01.071 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:00.777+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:01.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:00 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:00.745+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:01.405 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:01 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:11:01.262+0000 7fa74711c640 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:35.045410+0000 front 2024-09-19T04:10:35.045392+0000 (oldest deadline 2024-09-19T04:11:00.945558+0000) 2024-09-19T04:11:01.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:01 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:01.126+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:01.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:01 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:01.126+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:01.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:01 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:01.070+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:01.958 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:01 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:01.801+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:01.958 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:01 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:01.801+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:02.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:01 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:01.787+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:02.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:01 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:01.787+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:02.405 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:02 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:11:02.215+0000 7fa74711c640 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:35.045410+0000 front 2024-09-19T04:10:35.045392+0000 (oldest deadline 2024-09-19T04:11:00.945558+0000) 2024-09-19T04:11:02.405 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:02 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:02.251+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:02.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:02 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:02.137+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:02.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:02 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:02.137+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:02.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:02 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:02.092+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:02.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:02 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:02.092+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:03.084 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:02 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:02.826+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:03.085 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:02 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:02.826+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:03.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:02 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:02.738+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:03.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:02 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:02.738+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:03.405 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:03 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4[46727]: 2024-09-19T04:11:03.234+0000 7fa74711c640 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:35.045410+0000 front 2024-09-19T04:10:35.045392+0000 (oldest deadline 2024-09-19T04:11:00.945558+0000) 2024-09-19T04:11:03.405 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:03 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:03.293+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:03.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:03 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:03.146+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:03.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:03 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:03.146+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:03.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:03 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:03.083+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:03.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:03 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:03.083+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:03.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:03 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:03.272+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:04.094 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:03 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:03.828+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:04.094 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:03 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:03.828+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:04.181 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:03 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:03.691+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:04.182 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:03 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:03.698+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:04.182 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:03 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:03.698+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:04.405 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:04 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:04.287+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:04.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:04 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:04.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:04.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:04 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:04.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:04.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:04 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:04.276+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:04.459 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:04 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:04.093+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:04.459 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:04 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:04.093+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:04.716 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:04 smithi111 podman[68671]: 2024-09-19 04:11:04.412215018 +0000 UTC m=+5.453533834 container died 19198329ee78b3ac52a67bddaaaca4f2c6e8066283c619f584a1a72053cb4666 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T04:11:04.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:04.737+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:04.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:04.652+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:04.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:04 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:04.652+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:05.066 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:04 smithi111 podman[68671]: 2024-09-19 04:11:04.813917954 +0000 UTC m=+5.855236770 container cleanup 19198329ee78b3ac52a67bddaaaca4f2c6e8066283c619f584a1a72053cb4666 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T04:11:05.066 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:04 smithi111 bash[68671]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4 2024-09-19T04:11:05.112 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:04 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:04.856+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:05.113 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:04 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:04.856+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:05.405 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:05 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:05.284+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:05.405 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:05 smithi111 podman[68686]: 2024-09-19 04:11:05.065450618 +0000 UTC m=+0.656188039 container remove 19198329ee78b3ac52a67bddaaaca4f2c6e8066283c619f584a1a72053cb4666 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T04:11:05.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:05 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:05.106+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:05.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:05 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:05.106+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:05.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:05 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:05.111+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:05.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:05 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:05.111+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:05.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:05 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:05.241+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:05.795 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:05 smithi111 podman[68777]: 2024-09-19 04:11:05.389505621 +0000 UTC m=+0.015875847 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:05.795 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:05 smithi111 podman[68777]: 2024-09-19 04:11:05.499954301 +0000 UTC m=+0.126324565 container create f6273e287a36a8f4e59d5aa275c9f9f22b0c6d0e416ca3e096ffa1ac45df1db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:11:05.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:05 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:05.753+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:05.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:05 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:05.603+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:05.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:05 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:05.603+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:06.111 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:05 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:05.852+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:06.111 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:05 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:05.852+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:06.117 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:05 smithi111 podman[68777]: 2024-09-19 04:11:05.794297945 +0000 UTC m=+0.420668170 container init f6273e287a36a8f4e59d5aa275c9f9f22b0c6d0e416ca3e096ffa1ac45df1db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T04:11:06.117 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:05 smithi111 podman[68777]: 2024-09-19 04:11:05.797789879 +0000 UTC m=+0.424160101 container start f6273e287a36a8f4e59d5aa275c9f9f22b0c6d0e416ca3e096ffa1ac45df1db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T04:11:06.117 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:05 smithi111 podman[68777]: 2024-09-19 04:11:05.902690524 +0000 UTC m=+0.529060756 container attach f6273e287a36a8f4e59d5aa275c9f9f22b0c6d0e416ca3e096ffa1ac45df1db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:11:06.117 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 19 04:11:05 smithi111 podman[68777]: 2024-09-19 04:11:05.991244675 +0000 UTC m=+0.617614895 container died f6273e287a36a8f4e59d5aa275c9f9f22b0c6d0e416ca3e096ffa1ac45df1db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T04:11:06.405 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:06 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:06.319+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:06.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:06 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:06.115+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:06.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:06 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:06.115+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:06.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:06 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:06.246+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:06.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:06 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:06.109+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:06.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:06 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:06.109+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:06.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T04:11:06.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:11:06.746 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T04:11:06.747 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T04:11:06.747 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.5 2024-09-19T04:11:06.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:06 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:06.788+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:06.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:06 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:06.651+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:06.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:06 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:06.651+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:07.076 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:06 smithi196 systemd[1]: Stopping Ceph osd.5 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:11:07.076 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:06 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:06.891+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:07.076 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:06 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:06.891+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:07.405 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:07 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:07.291+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:07.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:07 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:07.067+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:07.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:07 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:07.067+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:07.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:07 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:07.206+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:07.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:07 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:07.074+0000 7fb230904640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:11:07.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:07 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:07.074+0000 7fb230904640 -1 osd.5 97 *** Got signal Terminated *** 2024-09-19T04:11:07.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:07 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:07.074+0000 7fb230904640 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:11:07.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:07 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:07.144+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:07.459 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:07 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:07.144+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:08.150 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:07 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:07.898+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:08.150 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:07 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:07.898+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:08.181 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:07 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:07.764+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:08.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:07 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:07.683+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:08.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:07 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:07.683+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:08.315 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:08 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:08.057+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:08.316 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:08 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:08.057+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:08.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:08 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:08.238+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:08.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:08 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:08.149+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:08.458 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:08 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:08.149+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:08.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:08 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:08.314+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:09.181 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:08 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:08.781+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:09.181 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:08 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:08.781+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:09.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:08 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:08.711+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:09.182 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:08 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:08.711+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:09.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:08 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:08.930+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:09.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:08 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:08.930+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:09.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:09.120+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:09.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:09.120+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:09.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:09.120+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.131365+0000 front 2024-09-19T04:10:46.131339+0000 (oldest deadline 2024-09-19T04:11:08.431506+0000) 2024-09-19T04:11:09.334 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:09.033+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:09.335 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:09.033+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:09.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:09 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:09.333+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:09.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:09.262+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:10.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:09.708+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:10.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:09.708+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:10.182 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:09.773+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:10.182 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:09.773+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:10.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:10 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:10.117+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:10.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:10 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:10.117+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:10.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:10 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:10.117+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.131365+0000 front 2024-09-19T04:10:46.131339+0000 (oldest deadline 2024-09-19T04:11:08.431506+0000) 2024-09-19T04:11:10.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:09.932+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:10.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:09.932+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:10.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:09 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:09.932+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:10.376 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:10 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:10.003+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:10.377 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:10 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:10.003+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:10.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:10 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:10.375+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:10.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:10 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:10.375+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:10.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:10 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:10.242+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:10.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:10 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:10.242+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:10.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:10.662+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:10.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:10.662+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:10.932 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:10.741+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:10.932 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:10 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:10.741+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:11.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:11 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:11.073+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.227536+0000 front 2024-09-19T04:10:28.227595+0000 (oldest deadline 2024-09-19T04:10:52.927610+0000) 2024-09-19T04:11:11.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:11 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:11.073+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.330270+0000 front 2024-09-19T04:10:37.330388+0000 (oldest deadline 2024-09-19T04:11:02.030407+0000) 2024-09-19T04:11:11.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:11 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5[46991]: 2024-09-19T04:11:11.073+0000 7fb228f12640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.131365+0000 front 2024-09-19T04:10:46.131339+0000 (oldest deadline 2024-09-19T04:11:08.431506+0000) 2024-09-19T04:11:11.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:10 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:10.914+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:11.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:10 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:10.914+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:11.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:10 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:10.914+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:11.370 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:11 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:11.048+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:11.371 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:11 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:11.048+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:11.371 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:11 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:11.048+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:11.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:11 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:11.369+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:11.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:11 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:11.369+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:11.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:11 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:11.292+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:11.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:11 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:11.292+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:12.181 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:11 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:11.781+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:12.182 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:11 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:11.781+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:12.182 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:11 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:11.681+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:12.182 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:11 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:11.681+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:12.182 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:11 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:11.681+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:12.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:11 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:11.928+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:12.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:11 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:11.928+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:12.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:11 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:11.928+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:12.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:12 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:12.049+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:12.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:12 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:12.049+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:12.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:12 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:12.049+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:12.591 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:12 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:12.250+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:12.591 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:12 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:12.250+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:12.892 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:12 smithi196 podman[69189]: 2024-09-19 04:11:12.590474583 +0000 UTC m=+5.645656324 container died f346cd81e199066781f174973eab876fb8115512677559b506b0f0c5f291c649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T04:11:12.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:12 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:12.417+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:12.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:12 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:12.417+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:12.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:12 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:12.799+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:12.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:12 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:12.799+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:12.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:12 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:12.633+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:12.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:12 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:12.633+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:12.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:12 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:12.633+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:13.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:12 smithi196 podman[69189]: 2024-09-19 04:11:12.991804152 +0000 UTC m=+6.046985890 container cleanup f346cd81e199066781f174973eab876fb8115512677559b506b0f0c5f291c649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T04:11:13.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:12 smithi196 bash[69189]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5 2024-09-19T04:11:13.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:12 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:12.943+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:13.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:12 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:12.943+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:13.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:12 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:12.943+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:13.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:13 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:13.066+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:13.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:13 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:13.066+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:13.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:13 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:13.066+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:13.562 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:13 smithi196 podman[69204]: 2024-09-19 04:11:13.23858033 +0000 UTC m=+0.636660486 container remove f346cd81e199066781f174973eab876fb8115512677559b506b0f0c5f291c649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:11:13.562 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:13 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:13.210+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:13.562 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:13 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:13.210+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:13.813 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:13 smithi196 podman[69294]: 2024-09-19 04:11:13.463811733 +0000 UTC m=+0.015940087 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:13.813 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:13 smithi196 podman[69294]: 2024-09-19 04:11:13.585207935 +0000 UTC m=+0.137336283 container create c746ac0c80ed764fa1f08d80548dd77f45b6e0924811c7e0bab79fb785fb5251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:11:13.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:13 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:13.432+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:13.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:13 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:13.432+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:13.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:13.633+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:13.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:13.633+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:13.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:13.633+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:13.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:13.766+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:13.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:13 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:13.766+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:14.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:13 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:13.989+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:14.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:13 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:13.989+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:14.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:13 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:13.989+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:14.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:13 smithi196 podman[69294]: 2024-09-19 04:11:13.812781744 +0000 UTC m=+0.364910080 container init c746ac0c80ed764fa1f08d80548dd77f45b6e0924811c7e0bab79fb785fb5251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:11:14.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:13 smithi196 podman[69294]: 2024-09-19 04:11:13.816188478 +0000 UTC m=+0.368316817 container start c746ac0c80ed764fa1f08d80548dd77f45b6e0924811c7e0bab79fb785fb5251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T04:11:14.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:13 smithi196 podman[69294]: 2024-09-19 04:11:13.896190987 +0000 UTC m=+0.448319323 container attach c746ac0c80ed764fa1f08d80548dd77f45b6e0924811c7e0bab79fb785fb5251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T04:11:14.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 04:11:14 smithi196 podman[69294]: 2024-09-19 04:11:14.002683457 +0000 UTC m=+0.554811796 container died c746ac0c80ed764fa1f08d80548dd77f45b6e0924811c7e0bab79fb785fb5251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T04:11:14.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:14 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:14.085+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:14.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:14 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:14.085+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:14.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:14 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:14.085+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:14.642 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:14 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:14.226+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:14.642 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:14 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:14.226+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:14.665 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T04:11:14.704 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:11:14.704 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T04:11:14.704 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T04:11:14.704 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.6 2024-09-19T04:11:14.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:14 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:14.477+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:14.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:14 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:14.477+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:14.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:14.591+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:14.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:14.591+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:14.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:14.591+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:14.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:14.751+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:14.932 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:14.751+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:14.932 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:14 smithi019 systemd[1]: Stopping Ceph osd.6 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:11:15.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:15 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:15.039+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:15.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:15 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:15.039+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:15.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:15 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:15.039+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:15.431 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:15.031+0000 7f687e3ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:11:15.431 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:15.031+0000 7f687e3ec640 -1 osd.6 97 *** Got signal Terminated *** 2024-09-19T04:11:15.431 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:15.031+0000 7f687e3ec640 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:11:15.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:15 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:15.219+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:15.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:15 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:15.219+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:15.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:15 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:15.040+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:15.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:15 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:15.040+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:15.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:15 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:15.040+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:15.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:15 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:15.480+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:15.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:15 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:15.480+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:15.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:15.593+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:15.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:15.593+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:15.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:15.593+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:15.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:15.718+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:15.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:15 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:15.718+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:16.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:16 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:16.024+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:16.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:16 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:16.024+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:16.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:16 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:16.024+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:16.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:16 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:16.264+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:16.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:16 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:16.264+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:16.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:16 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:16.073+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:16.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:16 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:16.073+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:16.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:16 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:16.073+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:16.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:16 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:16.434+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:16.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:16 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:16.434+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:16.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:16 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:16.677+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:16.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:16 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:16.677+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:16.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:16 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:16.609+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:16.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:16 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:16.609+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:16.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:16 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:16.609+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:17.309 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:17 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:17.050+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:17.309 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:17 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:17.050+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:17.309 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:17 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:17.050+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:17.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:17 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:17.074+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:17.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:17 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:17.074+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:17.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:17 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:17.074+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:17.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:17 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:17.308+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:17.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:17 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:17.308+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:17.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:17 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:17.308+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:17.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:17 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:17.420+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:17.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:17 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:17.420+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:17.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:17.642+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:17.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:17.642+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:17.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:17.628+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:17.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:17.628+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:17.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:17.628+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:17.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:17 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:17.628+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:18.268 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:18.002+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:18.268 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:18.002+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:18.269 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:18.002+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:18.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:18 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:18.119+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:18.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:18 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:18.119+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:18.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:18 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:18.119+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:18.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:18 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:18.119+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:18.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:18.267+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:18.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:18.267+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:18.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:18.267+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:18.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:18 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:18.413+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:18.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:18 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:18.413+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:18.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:18 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:18.630+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:18.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:18 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:18.630+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:18.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:18 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:18.646+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:18.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:18 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:18.646+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:18.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:18 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:18.646+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:18.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:18 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:18.646+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:19.252 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:18.975+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:19.252 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:18.975+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:19.252 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:18 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:18.975+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:19.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:19.097+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:19.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:19.097+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:19.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:19.097+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:19.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:19.097+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:19.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:19 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:19.250+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:19.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:19 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:19.250+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:19.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:19 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:19.250+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:19.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:19.456+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:19.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:19.456+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:19.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:19 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:19.456+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:19.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:19 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:19.673+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.399013+0000 front 2024-09-19T04:10:38.399155+0000 (oldest deadline 2024-09-19T04:11:03.098988+0000) 2024-09-19T04:11:19.931 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:19 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:19.673+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.500766+0000 front 2024-09-19T04:10:46.500977+0000 (oldest deadline 2024-09-19T04:11:08.200840+0000) 2024-09-19T04:11:19.932 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:19 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6[58039]: 2024-09-19T04:11:19.673+0000 7f687aa02640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.602389+0000 front 2024-09-19T04:10:54.602350+0000 (oldest deadline 2024-09-19T04:11:18.702498+0000) 2024-09-19T04:11:19.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:19 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:19.608+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:19.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:19 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:19.608+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:19.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:19 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:19.608+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:19.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:19 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:19.608+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:20.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:20 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:20.129+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:20.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:20 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:20.129+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:20.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:20 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:20.129+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:20.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:20 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:20.129+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:20.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:20.213+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:20.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:20.213+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:20.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:20.213+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:20.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:20.001+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:20.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:20.001+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:20.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:20.001+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:20.808 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:20 smithi019 podman[92460]: 2024-09-19 04:11:20.525505721 +0000 UTC m=+5.611376023 container died 54b3b724bf616d86a89ec5f4a94a2e8c25f9a571b5ec27e1d29303b934dabec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T04:11:20.808 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:20.631+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:20.808 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:20.631+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:20.808 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:20.631+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:20.808 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:20 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:20.631+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:20.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:20 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:20.422+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:20.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:20 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:20.422+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:20.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:20 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:20.422+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:21.181 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:20 smithi019 podman[92460]: 2024-09-19 04:11:20.99824022 +0000 UTC m=+6.084110551 container cleanup 54b3b724bf616d86a89ec5f4a94a2e8c25f9a571b5ec27e1d29303b934dabec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:11:21.181 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:20 smithi019 bash[92460]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6 2024-09-19T04:11:21.390 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:21.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:21.390 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:21.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:21.391 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:21.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:21.391 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:21.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:21.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:21 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:21.168+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:21.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:21 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:21.168+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:21.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:21 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:21.168+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:21.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:20.989+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:21.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:20.989+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:21.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:20 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:20.989+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:21.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:21.389+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:21.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:21.389+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:21.655 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:21 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:21.389+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:21.681 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:21 smithi019 podman[92475]: 2024-09-19 04:11:21.383509493 +0000 UTC m=+0.858295201 container remove 54b3b724bf616d86a89ec5f4a94a2e8c25f9a571b5ec27e1d29303b934dabec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T04:11:21.681 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:21 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:21.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:21.681 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:21 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:21.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:21.682 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:21 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:21.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:21.682 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:21 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:21.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:22.044 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:21 smithi019 podman[92565]: 2024-09-19 04:11:21.689980186 +0000 UTC m=+0.016223396 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:22.045 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:21 smithi019 podman[92565]: 2024-09-19 04:11:21.809444613 +0000 UTC m=+0.135687849 container create d14d0ff84f86ce3742df498108809d8b6d95a943dc3a08a1991ed0d50a882315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:11:22.311 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:22 smithi019 podman[92565]: 2024-09-19 04:11:22.161879967 +0000 UTC m=+0.488123171 container init d14d0ff84f86ce3742df498108809d8b6d95a943dc3a08a1991ed0d50a882315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T04:11:22.311 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:22 smithi019 podman[92565]: 2024-09-19 04:11:22.166245722 +0000 UTC m=+0.492488933 container start d14d0ff84f86ce3742df498108809d8b6d95a943dc3a08a1991ed0d50a882315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:11:22.312 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:22 smithi019 podman[92565]: 2024-09-19 04:11:22.245066748 +0000 UTC m=+0.571309956 container attach d14d0ff84f86ce3742df498108809d8b6d95a943dc3a08a1991ed0d50a882315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T04:11:22.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:22.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:22.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:22.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:22.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:22.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:22.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:22.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:22.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:22 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:22.121+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:22.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:22 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:22.121+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:22.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:22 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:22.121+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:22.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:22 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:22.021+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:22.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:22 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:22.021+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:22.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:22 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:22.021+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:22.654 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 19 04:11:22 smithi019 podman[92565]: 2024-09-19 04:11:22.366241317 +0000 UTC m=+0.692484530 container died d14d0ff84f86ce3742df498108809d8b6d95a943dc3a08a1991ed0d50a882315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T04:11:22.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:22.407+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:22.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:22.407+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:22.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:22.407+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:22.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:22 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:22.652+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:22.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:22 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:22.652+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:22.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:22 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:22.652+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:22.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:22 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:22.652+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:22.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T04:11:23.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:11:23.026 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T04:11:23.026 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T04:11:23.027 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.7 2024-09-19T04:11:23.319 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:23 smithi111 systemd[1]: Stopping Ceph osd.7 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:11:23.319 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:23.045+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:23.319 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:23.045+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:23.319 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:23.045+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:23.319 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:23.045+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:23.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:23 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:23.073+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:23.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:23 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:23.073+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:23.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:23 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:23.073+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:23.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:23 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:23.054+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:23.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:23 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:23.054+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:23.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:23 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:23.054+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:23.583 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:23.317+0000 7f699fe06640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:11:23.583 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:23.317+0000 7f699fe06640 -1 osd.7 97 *** Got signal Terminated *** 2024-09-19T04:11:23.583 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:23.317+0000 7f699fe06640 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:11:23.583 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:23.414+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:23.583 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:23.414+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:23.583 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:23 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:23.414+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:23.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:23 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:23.671+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:23.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:23 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:23.671+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:23.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:23 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:23.671+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:23.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:23 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:23.671+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:24.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:24 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:24.030+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:24.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:24 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:24.030+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:24.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:24 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:24.030+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:24.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:24 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:24.030+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:24.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:24 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:24.025+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:24.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:24 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:24.025+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:24.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:24 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:24.025+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:24.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:24 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:24.089+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:24.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:24 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:24.089+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:24.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:24 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:24.089+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:24.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:24 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:24.411+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:24.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:24 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:24.411+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:24.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:24 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:24.411+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:24.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:24 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:24.644+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:24.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:24 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:24.644+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:24.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:24 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:24.644+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:24.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:24 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:24.644+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:25.359 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:25.127+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:25.360 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:25.127+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:25.360 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:25.127+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:25.360 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:25.127+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:25.360 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:25.001+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:25.360 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:25.001+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:25.360 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:25.001+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:25.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:25.070+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:25.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:25.070+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:25.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:25.070+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:25.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:25.070+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:25.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:25.457+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:25.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:25.457+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:25.906 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:25 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:25.457+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:25.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:25.647+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:25.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:25.647+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:25.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:25.647+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:25.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:25 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:25.647+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:26.389 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:25.998+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:26.390 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:25.998+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:26.390 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:25.998+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:26.390 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:25 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:25.998+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:26.390 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:26 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:26.142+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:26.390 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:26 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:26.142+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:26.390 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:26 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:26.142+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:26.390 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:26 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:26.142+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:26.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:26.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:26.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:26.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:26.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:26.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:26.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:26.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:26.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:26.089+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:26.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:26.425+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:26.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:26.425+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:26.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:26.425+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:26.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:26.425+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.913559+0000 front 2024-09-19T04:11:03.913620+0000 (oldest deadline 2024-09-19T04:11:25.613803+0000) 2024-09-19T04:11:26.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:26 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:26.662+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:26.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:26 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:26.662+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:26.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:26 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:26.662+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:26.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:26 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:26.662+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:27.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:27.061+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:27.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:27.061+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:27.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:27.061+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:27.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:27.061+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:27.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:27.061+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:27.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:27.010+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:27.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:27.010+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:27.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:27.010+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:27.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:27.010+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:27.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:27.098+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:27.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:27.098+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:27.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:27.098+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:27.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:27.098+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:27.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:27.459+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.909926+0000 front 2024-09-19T04:10:38.909951+0000 (oldest deadline 2024-09-19T04:11:01.810076+0000) 2024-09-19T04:11:27.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:27.459+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.711003+0000 front 2024-09-19T04:10:44.711042+0000 (oldest deadline 2024-09-19T04:11:10.010984+0000) 2024-09-19T04:11:27.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:27.459+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:54.111857+0000 front 2024-09-19T04:10:54.111846+0000 (oldest deadline 2024-09-19T04:11:19.411824+0000) 2024-09-19T04:11:27.905 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:27 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7[53246]: 2024-09-19T04:11:27.459+0000 7f6998414640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.913559+0000 front 2024-09-19T04:11:03.913620+0000 (oldest deadline 2024-09-19T04:11:25.613803+0000) 2024-09-19T04:11:28.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:27 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:27.702+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:28.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:27 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:27.702+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:28.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:27 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:27.702+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:28.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:27 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:27.702+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:28.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:28 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:28.035+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:28.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:28 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:28.035+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:28.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:28 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:28.035+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:28.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:28 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:28.035+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:28.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:28 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:28.035+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:28.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:27.988+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:28.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:27.988+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:28.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:27.988+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:28.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:27 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:27.988+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:28.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:28.119+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:28.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:28.119+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:28.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:28.119+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:28.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:28.119+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:28.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:28 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:28.672+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:28.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:28 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:28.672+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:28.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:28 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:28.672+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:28.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:28 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:28.672+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:28.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:28 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:28.672+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:29.015 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:28 smithi111 podman[68881]: 2024-09-19 04:11:28.660871476 +0000 UTC m=+5.436241198 container died 91ae58c61dd618c8c75af42303872106f7293d9b9e0902980961f6c73cd748a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:11:29.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:28.944+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:29.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:28.944+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:29.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:28.944+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:29.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:28 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:28.944+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:29.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:29 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:29.101+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:29.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:29 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:29.101+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:29.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:29 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:29.101+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:29.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:29 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:29.101+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:29.405 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:29 smithi111 podman[68881]: 2024-09-19 04:11:29.117257703 +0000 UTC m=+5.892627422 container cleanup 91ae58c61dd618c8c75af42303872106f7293d9b9e0902980961f6c73cd748a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T04:11:29.405 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:29 smithi111 bash[68881]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7 2024-09-19T04:11:29.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:29 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:29.013+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:29.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:29 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:29.013+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:29.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:29 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:29.013+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:29.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:29 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:29.013+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:29.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:29 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:29.013+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:29.736 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:29 smithi111 podman[68908]: 2024-09-19 04:11:29.41017381 +0000 UTC m=+0.743453821 container remove 91ae58c61dd618c8c75af42303872106f7293d9b9e0902980961f6c73cd748a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T04:11:30.014 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:29 smithi111 podman[68996]: 2024-09-19 04:11:29.638217341 +0000 UTC m=+0.016104376 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:30.014 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:29 smithi111 podman[68996]: 2024-09-19 04:11:29.744266868 +0000 UTC m=+0.122153940 container create 99f6775930b76bb6f299817ad43cde792034d0bce97c84d11c0c0e092ed42586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T04:11:30.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:29 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:29.718+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:30.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:29 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:29.718+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:30.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:29 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:29.718+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:30.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:29 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:29.718+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:30.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:29 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:29.718+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:30.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:30 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:30.041+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:30.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:30 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:30.041+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:30.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:30 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:30.041+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:30.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:30 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:30.041+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:30.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:30 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:30.041+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:30.406 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:30 smithi111 podman[68996]: 2024-09-19 04:11:30.013939524 +0000 UTC m=+0.391826557 container init 99f6775930b76bb6f299817ad43cde792034d0bce97c84d11c0c0e092ed42586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T04:11:30.406 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:30 smithi111 podman[68996]: 2024-09-19 04:11:30.017271274 +0000 UTC m=+0.395158305 container start 99f6775930b76bb6f299817ad43cde792034d0bce97c84d11c0c0e092ed42586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T04:11:30.406 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:30 smithi111 podman[68996]: 2024-09-19 04:11:30.13883539 +0000 UTC m=+0.516722436 container attach 99f6775930b76bb6f299817ad43cde792034d0bce97c84d11c0c0e092ed42586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T04:11:30.406 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 19 04:11:30 smithi111 podman[68996]: 2024-09-19 04:11:30.206801458 +0000 UTC m=+0.584688489 container died 99f6775930b76bb6f299817ad43cde792034d0bce97c84d11c0c0e092ed42586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:11:30.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:29 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:29.959+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:30.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:29 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:29.959+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:30.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:29 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:29.959+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:30.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:29 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:29.959+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:30.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:30.078+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:30.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:30.079+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:30.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:30.079+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:30.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:30.079+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:30.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:30 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:30.670+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:30.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:30 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:30.670+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:30.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:30 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:30.670+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:30.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:30 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:30.670+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:30.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:30 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:30.670+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:30.945 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T04:11:30.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:11:30.988 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T04:11:30.988 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T04:11:30.988 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.8 2024-09-19T04:11:31.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:31.083+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:31.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:31.083+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:31.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:31.083+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:31.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:31.083+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:31.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:30.946+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:31.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:30.946+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:31.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:30.946+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:31.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:30 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:30.946+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:31.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:31 smithi196 systemd[1]: Stopping Ceph osd.8 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:11:31.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:31 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:31.074+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:31.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:31 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:31.074+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:31.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:31 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:31.074+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:31.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:31 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:31.074+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:31.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:31 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:31.074+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:31.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:31.301+0000 7f30134d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:11:31.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:31.301+0000 7f30134d7640 -1 osd.8 97 *** Got signal Terminated *** 2024-09-19T04:11:31.708 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:31.301+0000 7f30134d7640 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:11:31.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:31 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:31.624+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:31.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:31 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:31.624+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:31.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:31 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:31.624+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:31.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:31 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:31.624+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:31.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:31 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:31.624+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:32.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:31.939+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:32.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:31.939+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:32.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:31.939+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:32.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:31 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:31.939+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:32.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:32 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:32.114+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:32.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:32 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:32.114+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:32.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:32 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:32.114+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:32.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:32 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:32.114+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:32.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:32 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:32.095+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:32.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:32 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:32.095+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:32.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:32 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:32.095+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:32.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:32 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:32.095+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:32.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:32 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:32.095+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:32.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:32 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:32.599+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:32.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:32 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:32.599+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:32.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:32 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:32.599+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:32.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:32 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:32.599+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:32.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:32 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:32.599+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:33.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:33 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:33.139+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:33.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:33 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:33.139+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:33.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:33 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:33.139+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:33.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:33 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:33.139+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:33.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:33 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:33.139+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:33.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:32 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:32.968+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:33.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:32 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:32.968+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:33.458 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:32 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:32.968+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:33.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:32 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:32.968+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:33.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:33 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:33.162+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:33.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:33 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:33.162+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:33.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:33 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:33.162+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:33.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:33 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:33.162+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:33.681 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:33 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:33.590+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:33.681 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:33 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:33.590+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:33.681 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:33 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:33.590+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:33.682 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:33 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:33.590+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:33.682 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:33 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:33.590+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:33.682 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:33 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:33.590+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:34.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:34.151+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:34.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:34.151+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:34.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:34.151+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:34.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:34.151+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:34.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:33 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:33.929+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:34.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:33 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:33.929+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:34.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:33 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:33.929+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:34.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:33 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:33.929+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:34.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:34 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:34.161+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:34.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:34 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:34.161+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:34.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:34 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:34.161+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:34.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:34 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:34.161+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:34.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:34 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:34.161+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:34.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:34 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:34.636+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:34.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:34 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:34.636+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:34.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:34 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:34.636+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:34.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:34 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:34.636+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:34.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:34 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:34.636+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:34.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:34 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:34.636+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:35.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:35.188+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:35.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:35.188+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:35.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:35.188+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:35.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:35.188+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:35.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:34.963+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:35.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:34.963+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:35.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:34.963+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:35.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:34.963+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:35.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:35 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:35.180+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:35.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:35 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:35.180+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:35.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:35 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:35.180+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:35.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:35 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:35.180+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:35.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:35 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:35.180+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:35.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:35 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:35.180+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:35.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:35.648+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:35.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:35.648+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:35.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:35.648+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:35.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:35.648+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:35.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:35.648+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:35.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:35 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:35.648+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:36.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:36 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:36.134+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:36.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:36 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:36.134+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:36.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:36 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:36.134+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:36.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:36 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:36.134+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:36.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:36 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:36.134+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:36.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:36 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:36.134+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:36.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:36 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:36.164+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:36.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:36 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:36.164+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:36.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:36 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:36.164+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:36.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:36 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:36.164+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:36.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:35.995+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.005056+0000 front 2024-09-19T04:10:37.005075+0000 (oldest deadline 2024-09-19T04:11:02.905063+0000) 2024-09-19T04:11:36.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:35.995+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:45.805787+0000 front 2024-09-19T04:10:45.805741+0000 (oldest deadline 2024-09-19T04:11:09.323101+0000) 2024-09-19T04:11:36.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:35.995+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:55.224090+0000 front 2024-09-19T04:10:55.224031+0000 (oldest deadline 2024-09-19T04:11:16.923954+0000) 2024-09-19T04:11:36.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:35.995+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:02.725581+0000 front 2024-09-19T04:11:02.725652+0000 (oldest deadline 2024-09-19T04:11:25.025413+0000) 2024-09-19T04:11:36.459 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:35 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8[53932]: 2024-09-19T04:11:35.995+0000 7f300bae5640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.426988+0000 front 2024-09-19T04:11:11.427067+0000 (oldest deadline 2024-09-19T04:11:35.527173+0000) 2024-09-19T04:11:36.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:36.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:36.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:36.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:36.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:36.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:36.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:36.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:36.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:36.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:36.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:36 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:36.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:36.953 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:36 smithi196 podman[69408]: 2024-09-19 04:11:36.664310514 +0000 UTC m=+5.477699946 container died 0652d28061068c19cf22692c2db81e72122d2bd62976eb9b93f1cf7c5856dcce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T04:11:37.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:37 smithi196 podman[69408]: 2024-09-19 04:11:37.059939333 +0000 UTC m=+5.873328755 container cleanup 0652d28061068c19cf22692c2db81e72122d2bd62976eb9b93f1cf7c5856dcce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T04:11:37.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:37 smithi196 bash[69408]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8 2024-09-19T04:11:37.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:37.199+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:37.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:37.199+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:37.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:37.199+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:37.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:37.199+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:37.209 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:37 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:37.199+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:37.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:37 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:37.138+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:37.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:37 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:37.138+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:37.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:37 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:37.138+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:37.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:37 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:37.138+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:37.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:37 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:37.138+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:37.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:37 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:37.138+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:37.627 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:37 smithi196 podman[69423]: 2024-09-19 04:11:37.311190928 +0000 UTC m=+0.646975037 container remove 0652d28061068c19cf22692c2db81e72122d2bd62976eb9b93f1cf7c5856dcce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T04:11:37.915 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:37 smithi196 podman[69509]: 2024-09-19 04:11:37.528931624 +0000 UTC m=+0.015474059 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:37.915 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:37 smithi196 podman[69509]: 2024-09-19 04:11:37.658494225 +0000 UTC m=+0.145036641 container create 1760f68f0db006d13717a58c202b43e168e4f64e296b6733067a3d003d61eba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T04:11:37.915 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:37 smithi196 podman[69509]: 2024-09-19 04:11:37.914694646 +0000 UTC m=+0.401237061 container init 1760f68f0db006d13717a58c202b43e168e4f64e296b6733067a3d003d61eba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T04:11:37.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:37.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:37.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:37.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:37.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:37.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:37.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:37.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:37.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:37.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:37.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:37 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:37.620+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:38.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:37 smithi196 podman[69509]: 2024-09-19 04:11:37.918190413 +0000 UTC m=+0.404732830 container start 1760f68f0db006d13717a58c202b43e168e4f64e296b6733067a3d003d61eba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T04:11:38.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:37 smithi196 podman[69509]: 2024-09-19 04:11:37.998065029 +0000 UTC m=+0.484607457 container attach 1760f68f0db006d13717a58c202b43e168e4f64e296b6733067a3d003d61eba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T04:11:38.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 04:11:38 smithi196 podman[69509]: 2024-09-19 04:11:38.104204969 +0000 UTC m=+0.590747397 container died 1760f68f0db006d13717a58c202b43e168e4f64e296b6733067a3d003d61eba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:11:38.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:38 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:38.121+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:38.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:38 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:38.121+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:38.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:38 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:38.121+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:38.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:38 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:38.121+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:38.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:38 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:38.121+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:38.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:38 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:38.121+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:38.595 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:38.243+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:38.596 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:38.243+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:38.596 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:38.243+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:38.596 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:38.243+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:38.596 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:38.243+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:38.777 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T04:11:38.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:11:38.816 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T04:11:38.816 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T04:11:38.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.9 2024-09-19T04:11:38.845 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:38.585+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:38.845 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:38.585+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:38.845 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:38.585+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:38.845 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:38.585+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:38.845 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:38.585+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:38.845 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:38 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:38.585+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:39.102 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:38 smithi019 systemd[1]: Stopping Ceph osd.9 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:11:39.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:39.151+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:39.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:39.151+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:39.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:39.151+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:39.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:39.151+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:39.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:39.151+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:39.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:39 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:39.151+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:39.431 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.099+0000 7fb69b538640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:11:39.431 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.099+0000 7fb69b538640 -1 osd.9 97 *** Got signal Terminated *** 2024-09-19T04:11:39.431 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.099+0000 7fb69b538640 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:11:39.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:39 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:39.273+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:39.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:39 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:39.273+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:39.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:39 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:39.273+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:39.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:39 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:39.273+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:39.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:39 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:39.273+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:39.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:39.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:39.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:39.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:39.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:39.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:39 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:39.618+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:40.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:40.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:40.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:40.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:40.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:40.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:40.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:40.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:40.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:40.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:40.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:40 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:40.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:40.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:40.279+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:40.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:40.279+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:40.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:40.279+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:40.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:40.279+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:40.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:40 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:40.279+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:40.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:40 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:40.616+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:40.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:40 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:40.616+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:40.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:40 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:40.616+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:40.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:40 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:40.616+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:40.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:40 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:40.616+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:40.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:40 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:40.616+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:41.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:41 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:41.196+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:41.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:41 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:41.196+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:41.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:41 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:41.196+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:41.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:41 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:41.196+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:41.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:41 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:41.196+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:41.656 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:41 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:41.196+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:41.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:41.230+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:41.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:41.230+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:41.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:41.230+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:41.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:41.230+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:41.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:41 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:41.230+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:41.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:41 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:41.573+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:41.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:41 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:41.573+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:41.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:41 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:41.573+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:41.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:41 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:41.573+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:41.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:41 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:41.573+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:41.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:41 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:41.573+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:42.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:42.147+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:42.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:42.147+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:42.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:42.147+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:42.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:42.147+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:42.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:42.147+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:42.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:42.147+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:42.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:42 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:42.147+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:42.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:42.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:42.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:42.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:42.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:42.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:42.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:42.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:42.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:42.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:42.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:42.540+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:42.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:42.540+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:42.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:42.540+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:42.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:42.540+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:42.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:42.540+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:42.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:42 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:42.540+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:43.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:43.130+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:43.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:43.130+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:43.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:43.130+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:43.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:43.130+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:43.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:43.130+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:43.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:43.130+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:43.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:43 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:43.130+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:43.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:43 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:43.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:43.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:43 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:43.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:43.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:43 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:43.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:43.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:43 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:43.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:43.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:43 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:43.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:43.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:43 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:43.513+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:26.498597+0000 front 2024-09-19T04:10:26.498557+0000 (oldest deadline 2024-09-19T04:10:57.099238+0000) 2024-09-19T04:11:43.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:43 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:43.513+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:38.800396+0000 front 2024-09-19T04:10:38.800408+0000 (oldest deadline 2024-09-19T04:11:01.700413+0000) 2024-09-19T04:11:43.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:43 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:43.513+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:47.402630+0000 front 2024-09-19T04:10:47.402476+0000 (oldest deadline 2024-09-19T04:11:10.902301+0000) 2024-09-19T04:11:43.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:43 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:43.513+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.002979+0000 front 2024-09-19T04:10:52.002946+0000 (oldest deadline 2024-09-19T04:11:17.303107+0000) 2024-09-19T04:11:43.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:43 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:43.513+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:11:03.203880+0000 front 2024-09-19T04:11:03.203947+0000 (oldest deadline 2024-09-19T04:11:28.503939+0000) 2024-09-19T04:11:43.932 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:43 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9[64699]: 2024-09-19T04:11:43.513+0000 7fb697b4e640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:08.504644+0000 front 2024-09-19T04:11:08.504732+0000 (oldest deadline 2024-09-19T04:11:33.204433+0000) 2024-09-19T04:11:44.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:44 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:44.117+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:44.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:44 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:44.117+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:44.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:44 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:44.117+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:44.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:44 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:44.117+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:44.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:44 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:44.117+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:44.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:44 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:44.117+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:44.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:44 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:44.117+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:44.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:44 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:44.251+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:44.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:44 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:44.251+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:44.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:44 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:44.251+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:44.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:44 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:44.251+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:44.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:44 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:44.251+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:44.931 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:44 smithi019 podman[92682]: 2024-09-19 04:11:44.605181559 +0000 UTC m=+5.590125662 container died a152060d7b29fcf42a83657fb2b8cb49c8d38f0efba98a2568a2779008e69eb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T04:11:45.327 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:45 smithi019 podman[92682]: 2024-09-19 04:11:45.133477336 +0000 UTC m=+6.118421439 container cleanup a152060d7b29fcf42a83657fb2b8cb49c8d38f0efba98a2568a2779008e69eb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T04:11:45.327 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:45 smithi019 bash[92682]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9 2024-09-19T04:11:45.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:45 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:45.205+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:45.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:45 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:45.205+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:45.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:45 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:45.205+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:45.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:45 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:45.205+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:45.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:45 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:45.205+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:45.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:45 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:45.206+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:45.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:45 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:45.163+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:45.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:45 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:45.163+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:45.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:45 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:45.163+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:45.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:45 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:45.163+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:45.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:45 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:45.163+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:45.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:45 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:45.163+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:45.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:45 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:45.163+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:45.681 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:45 smithi019 podman[92698]: 2024-09-19 04:11:45.326474611 +0000 UTC m=+0.715057590 container remove a152060d7b29fcf42a83657fb2b8cb49c8d38f0efba98a2568a2779008e69eb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T04:11:46.080 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:45 smithi019 podman[92785]: 2024-09-19 04:11:45.658002932 +0000 UTC m=+0.016434677 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:46.080 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:45 smithi019 podman[92785]: 2024-09-19 04:11:45.777698207 +0000 UTC m=+0.136129942 container create 38776f2a4f771658a44ca98e6af938299f848f0e77d9035f53b84bc6a93621c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T04:11:46.081 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:45 smithi019 podman[92785]: 2024-09-19 04:11:45.996704512 +0000 UTC m=+0.355136257 container init 38776f2a4f771658a44ca98e6af938299f848f0e77d9035f53b84bc6a93621c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T04:11:46.081 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:46 smithi019 podman[92785]: 2024-09-19 04:11:46.000055635 +0000 UTC m=+0.358487374 container start 38776f2a4f771658a44ca98e6af938299f848f0e77d9035f53b84bc6a93621c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T04:11:46.081 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:46 smithi019 podman[92785]: 2024-09-19 04:11:46.079678239 +0000 UTC m=+0.438109983 container attach 38776f2a4f771658a44ca98e6af938299f848f0e77d9035f53b84bc6a93621c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T04:11:46.354 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 19 04:11:46 smithi019 podman[92785]: 2024-09-19 04:11:46.202345537 +0000 UTC m=+0.560777272 container died 38776f2a4f771658a44ca98e6af938299f848f0e77d9035f53b84bc6a93621c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:11:46.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:46 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:46.153+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:46.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:46 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:46.153+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:46.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:46 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:46.153+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:46.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:46 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:46.153+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:46.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:46 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:46.153+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:46.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:46 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:46.153+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:46.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:46 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:46.153+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:46.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:46 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:46.255+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:46.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:46 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:46.255+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:46.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:46 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:46.255+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:46.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:46 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:46.255+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:46.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:46 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:46.255+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:46.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:46 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:46.255+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:47.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T04:11:47.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:11:47.115 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T04:11:47.115 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T04:11:47.115 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.10 2024-09-19T04:11:47.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 systemd[1]: Stopping Ceph osd.10 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:11:47.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:47.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:47.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:47.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:47.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:47.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:47.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.178+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:47.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:47 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:47.282+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:47.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:47 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:47.282+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:47.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:47 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:47.282+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:47.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:47 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:47.282+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:47.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:47 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:47.282+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:47.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:47 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:47.282+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:47.905 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.427+0000 7f0adb732640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:11:47.905 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.427+0000 7f0adb732640 -1 osd.10 97 *** Got signal Terminated *** 2024-09-19T04:11:47.905 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:47 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:47.427+0000 7f0adb732640 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:11:48.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:48.162+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:48.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:48.162+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:48.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:48.162+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:48.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:48.162+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:48.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:48.162+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:48.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:48.162+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:48.656 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:48 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:48.162+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:48.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:48 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:48.262+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:48.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:48 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:48.262+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:48.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:48 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:48.262+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:48.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:48 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:48.262+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:48.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:48 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:48.262+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:48.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:48 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:48.262+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:49.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:49 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:49.154+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:49.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:49 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:49.154+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:49.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:49 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:49.154+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:49.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:49 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:49.154+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:49.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:49 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:49.154+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:49.655 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:49 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:49.154+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:49.656 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:49 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:49.154+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:49.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:49 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:49.267+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:49.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:49 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:49.267+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:49.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:49 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:49.267+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:49.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:49 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:49.267+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:49.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:49 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:49.267+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:49.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:49 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:49.267+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:50.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:50 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:50.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:50.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:50 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:50.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:50.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:50 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:50.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:50.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:50 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:50.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:50.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:50 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:50.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:50.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:50 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:50.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:50.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:50 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:50.116+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:50.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:50 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:50.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:50.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:50 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:50.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:50.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:50 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:50.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:50.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:50 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:50.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:50.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:50 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:50.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:50.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:50 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:50.252+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:51.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:51 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:51.148+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:51.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:51 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:51.148+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:51.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:51 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:51.148+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:51.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:51 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:51.148+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:51.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:51 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:51.148+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:51.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:51 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:51.148+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:51.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:51 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:51.148+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:51.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:51 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:51.247+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:51.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:51 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:51.247+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:51.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:51 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:51.247+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:51.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:51 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:51.247+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:51.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:51 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:51.247+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:51.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:51 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:51.247+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:51.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:51 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:51.247+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:52.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:52 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:52.120+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:28.872461+0000 front 2024-09-19T04:10:28.872613+0000 (oldest deadline 2024-09-19T04:10:54.172390+0000) 2024-09-19T04:11:52.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:52 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:52.120+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:34.673187+0000 front 2024-09-19T04:10:34.673127+0000 (oldest deadline 2024-09-19T04:11:00.573255+0000) 2024-09-19T04:11:52.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:52 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:52.120+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:46.374679+0000 front 2024-09-19T04:10:46.374927+0000 (oldest deadline 2024-09-19T04:11:10.474644+0000) 2024-09-19T04:11:52.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:52 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:52.120+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-19T04:10:52.175578+0000 front 2024-09-19T04:10:52.175667+0000 (oldest deadline 2024-09-19T04:11:17.475406+0000) 2024-09-19T04:11:52.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:52 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:52.120+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:59.176037+0000 front 2024-09-19T04:10:59.176009+0000 (oldest deadline 2024-09-19T04:11:25.076094+0000) 2024-09-19T04:11:52.405 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:52 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:52.120+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:09.677551+0000 front 2024-09-19T04:11:09.677655+0000 (oldest deadline 2024-09-19T04:11:34.377450+0000) 2024-09-19T04:11:52.406 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:52 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10[60107]: 2024-09-19T04:11:52.120+0000 7f0ad7d48640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:15.379108+0000 front 2024-09-19T04:11:15.379127+0000 (oldest deadline 2024-09-19T04:11:41.278855+0000) 2024-09-19T04:11:52.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:52 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:52.265+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:52.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:52 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:52.265+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:52.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:52 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:52.265+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:52.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:52 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:52.265+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:52.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:52 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:52.265+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:52.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:52 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:52.265+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:52.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:52 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:52.265+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:53.221 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:52 smithi111 podman[69099]: 2024-09-19 04:11:52.953732885 +0000 UTC m=+5.645014078 container died 589f208e62ea89870fb6ba6ae45e6e02c8c60729fa1513bbfed2162baaba86b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T04:11:53.571 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:53 smithi111 podman[69099]: 2024-09-19 04:11:53.319328084 +0000 UTC m=+6.010609278 container cleanup 589f208e62ea89870fb6ba6ae45e6e02c8c60729fa1513bbfed2162baaba86b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T04:11:53.571 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:53 smithi111 bash[69099]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10 2024-09-19T04:11:53.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:53 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:53.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:53.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:53 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:53.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:53.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:53 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:53.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:53.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:53 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:53.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:53.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:53 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:53.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:53.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:53 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:53.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:53.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:53 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:53.257+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:53.890 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:53 smithi111 podman[69114]: 2024-09-19 04:11:53.570562538 +0000 UTC m=+0.611695327 container remove 589f208e62ea89870fb6ba6ae45e6e02c8c60729fa1513bbfed2162baaba86b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T04:11:54.155 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:53 smithi111 podman[69206]: 2024-09-19 04:11:53.792554632 +0000 UTC m=+0.016500955 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:11:54.155 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:53 smithi111 podman[69206]: 2024-09-19 04:11:53.912917521 +0000 UTC m=+0.136863826 container create 4ddfe447c5e82a7eb375ece6c5e6d6410b56fc9924be9075c7d2e97193d9396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T04:11:54.155 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:54 smithi111 podman[69206]: 2024-09-19 04:11:54.131828193 +0000 UTC m=+0.355774497 container init 4ddfe447c5e82a7eb375ece6c5e6d6410b56fc9924be9075c7d2e97193d9396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T04:11:54.155 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:54 smithi111 podman[69206]: 2024-09-19 04:11:54.135063801 +0000 UTC m=+0.359010113 container start 4ddfe447c5e82a7eb375ece6c5e6d6410b56fc9924be9075c7d2e97193d9396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T04:11:54.627 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:54 smithi111 podman[69206]: 2024-09-19 04:11:54.2069124 +0000 UTC m=+0.430858719 container attach 4ddfe447c5e82a7eb375ece6c5e6d6410b56fc9924be9075c7d2e97193d9396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:11:54.627 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 19 04:11:54 smithi111 podman[69206]: 2024-09-19 04:11:54.324644717 +0000 UTC m=+0.548591038 container died 4ddfe447c5e82a7eb375ece6c5e6d6410b56fc9924be9075c7d2e97193d9396b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T04:11:54.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:54.303+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:54.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:54.303+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:54.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:54.303+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:54.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:54.303+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:54.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:54.303+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:54.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:54.303+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:54.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:54 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:54.303+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:54.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T04:11:54.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:11:54.958 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T04:11:54.958 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T04:11:54.958 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.11 2024-09-19T04:11:55.271 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 systemd[1]: Stopping Ceph osd.11 for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.269+0000 7f59a6641640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.269+0000 7f59a6641640 -1 osd.11 97 *** Got signal Terminated *** 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.269+0000 7f59a6641640 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.299+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.299+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.299+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.299+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.299+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:55.526 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.299+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:55.527 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:55 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:55.299+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:56.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:56.285+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:56.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:56.285+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:56.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:56.285+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:56.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:56.285+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:56.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:56.285+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:56.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:56.285+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:56.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:56 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:56.285+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:57.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:57.258+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:57.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:57.258+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:57.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:57.258+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:57.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:57.258+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:57.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:57.258+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:57.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:57.258+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:57.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:57 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:57.258+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:58.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:58.225+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:58.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:58.225+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:58.708 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:58.225+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:58.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:58.225+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:58.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:58.225+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:58.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:58.225+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:58.709 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:58 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:58.225+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:59.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:59.182+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:11:59.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:59.182+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:11:59.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:59.182+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:11:59.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:59.182+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:11:59.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:59.182+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:11:59.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:59.182+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:11:59.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:59.182+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:11:59.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:11:59 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:11:59.182+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-09-19T04:11:36.269951+0000 front 2024-09-19T04:11:36.269970+0000 (oldest deadline 2024-09-19T04:11:58.570076+0000) 2024-09-19T04:12:00.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:12:00.190+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-19T04:10:29.561042+0000 front 2024-09-19T04:10:29.561116+0000 (oldest deadline 2024-09-19T04:10:55.460996+0000) 2024-09-19T04:12:00.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:12:00.190+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-19T04:10:37.762182+0000 front 2024-09-19T04:10:37.762080+0000 (oldest deadline 2024-09-19T04:11:00.662062+0000) 2024-09-19T04:12:00.458 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:12:00.190+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T04:10:44.063449+0000 front 2024-09-19T04:10:44.063482+0000 (oldest deadline 2024-09-19T04:11:09.363486+0000) 2024-09-19T04:12:00.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:12:00.190+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-19T04:10:58.765266+0000 front 2024-09-19T04:10:58.765140+0000 (oldest deadline 2024-09-19T04:11:24.665155+0000) 2024-09-19T04:12:00.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:12:00.190+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T04:11:11.666800+0000 front 2024-09-19T04:11:11.666838+0000 (oldest deadline 2024-09-19T04:11:36.966976+0000) 2024-09-19T04:12:00.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:12:00.190+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-19T04:11:19.868014+0000 front 2024-09-19T04:11:19.867991+0000 (oldest deadline 2024-09-19T04:11:45.167885+0000) 2024-09-19T04:12:00.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:12:00.190+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-19T04:11:25.168542+0000 front 2024-09-19T04:11:25.168418+0000 (oldest deadline 2024-09-19T04:11:50.468287+0000) 2024-09-19T04:12:00.459 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11[60664]: 2024-09-19T04:12:00.190+0000 7f59a2c57640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-09-19T04:11:36.269951+0000 front 2024-09-19T04:11:36.269970+0000 (oldest deadline 2024-09-19T04:11:58.570076+0000) 2024-09-19T04:12:00.855 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 podman[69626]: 2024-09-19 04:12:00.572855208 +0000 UTC m=+5.409077970 container died a688b07d0cfbfa36e2dee5ba04ba0c23762f6b621edb930efe8607913cf679d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:12:01.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 podman[69626]: 2024-09-19 04:12:00.952845841 +0000 UTC m=+5.789068600 container cleanup a688b07d0cfbfa36e2dee5ba04ba0c23762f6b621edb930efe8607913cf679d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T04:12:01.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:00 smithi196 bash[69626]: ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11 2024-09-19T04:12:01.543 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:01 smithi196 podman[69653]: 2024-09-19 04:12:01.237329549 +0000 UTC m=+0.655647471 container remove a688b07d0cfbfa36e2dee5ba04ba0c23762f6b621edb930efe8607913cf679d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T04:12:01.875 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:01 smithi196 podman[69739]: 2024-09-19 04:12:01.445833869 +0000 UTC m=+0.015801329 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:12:01.875 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:01 smithi196 podman[69739]: 2024-09-19 04:12:01.546517331 +0000 UTC m=+0.116484830 container create a50abd7f0cab67fe7d9fc6858f2b67daa11fb1feb2492362ccb1700f9574e9df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T04:12:01.875 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:01 smithi196 podman[69739]: 2024-09-19 04:12:01.782534351 +0000 UTC m=+0.352501801 container init a50abd7f0cab67fe7d9fc6858f2b67daa11fb1feb2492362ccb1700f9574e9df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T04:12:01.875 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:01 smithi196 podman[69739]: 2024-09-19 04:12:01.785693747 +0000 UTC m=+0.355661200 container start a50abd7f0cab67fe7d9fc6858f2b67daa11fb1feb2492362ccb1700f9574e9df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T04:12:01.875 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:01 smithi196 podman[69739]: 2024-09-19 04:12:01.874238657 +0000 UTC m=+0.444206121 container attach a50abd7f0cab67fe7d9fc6858f2b67daa11fb1feb2492362ccb1700f9574e9df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T04:12:02.208 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:01 smithi196 podman[69739]: 2024-09-19 04:12:01.976517446 +0000 UTC m=+0.546484907 container died a50abd7f0cab67fe7d9fc6858f2b67daa11fb1feb2492362ccb1700f9574e9df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T04:12:02.519 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T04:12:02.542 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:02 smithi196 podman[69762]: 2024-09-19 04:12:02.502626024 +0000 UTC m=+0.516460325 container remove a50abd7f0cab67fe7d9fc6858f2b67daa11fb1feb2492362ccb1700f9574e9df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T04:12:02.542 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:02 smithi196 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-19T04:12:02.542 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:02 smithi196 systemd[1]: Stopped Ceph osd.11 for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:12:02.542 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 04:12:02 smithi196 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@osd.11.service: Consumed 10.385s CPU time. 2024-09-19T04:12:02.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:02.557 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T04:12:02.557 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T04:12:02.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.a 2024-09-19T04:12:02.872 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:12:02 smithi019 systemd[1]: Stopping Ceph mds.a for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:03.182 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:12:02 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a[71574]: 2024-09-19T04:12:02.868+0000 7fe44c415640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:03.182 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:12:02 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a[71574]: 2024-09-19T04:12:02.868+0000 7fe44c415640 -1 mds.a *** got signal Terminated *** 2024-09-19T04:12:05.586 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:12:05 smithi019 podman[92902]: 2024-09-19 04:12:05.311813619 +0000 UTC m=+2.555400680 container died d719fb1c5e70bda387eb33d34edd17cb0a42eb535e13d4f1c00a426fb70e485f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-19T04:12:05.931 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:12:05 smithi019 podman[92902]: 2024-09-19 04:12:05.760208519 +0000 UTC m=+3.003795579 container cleanup d719fb1c5e70bda387eb33d34edd17cb0a42eb535e13d4f1c00a426fb70e485f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T04:12:05.932 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 19 04:12:05 smithi019 bash[92902]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-a 2024-09-19T04:12:06.205 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T04:12:06.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:06.241 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T04:12:06.241 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T04:12:06.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.d 2024-09-19T04:12:06.565 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:12:06 smithi019 systemd[1]: Stopping Ceph mds.d for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:06.931 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:12:06 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d[70393]: 2024-09-19T04:12:06.564+0000 7fafbecc1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:06.931 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:12:06 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d[70393]: 2024-09-19T04:12:06.564+0000 7fafbecc1640 -1 mds.d *** got signal Terminated *** 2024-09-19T04:12:08.382 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:12:08 smithi019 podman[93022]: 2024-09-19 04:12:08.122494371 +0000 UTC m=+1.657658736 container died a4342fddd53d162df38821c1467221bef1f8e88a60f4cd1d4194a494f8e80fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T04:12:08.681 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:12:08 smithi019 podman[93022]: 2024-09-19 04:12:08.572455098 +0000 UTC m=+2.107619463 container cleanup a4342fddd53d162df38821c1467221bef1f8e88a60f4cd1d4194a494f8e80fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T04:12:08.681 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 19 04:12:08 smithi019 bash[93022]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-d 2024-09-19T04:12:09.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T04:12:09.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:09.047 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T04:12:09.047 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T04:12:09.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.g 2024-09-19T04:12:09.394 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:09 smithi019 systemd[1]: Stopping Ceph mds.g for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:09.681 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-g[71271]: 2024-09-19T04:12:09.392+0000 7f63b6bbd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:09.681 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:09 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-g[71271]: 2024-09-19T04:12:09.392+0000 7f63b6bbd640 -1 mds.g *** got signal Terminated *** 2024-09-19T04:12:13.595 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:13 smithi019 podman[93142]: 2024-09-19 04:12:13.30317088 +0000 UTC m=+4.027411119 container died 2d75d5ec4ee0eeb2ac34136bd8db10cccef7b38d9530937752ef19964258e4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T04:12:13.886 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:13 smithi019 podman[93142]: 2024-09-19 04:12:13.751253213 +0000 UTC m=+4.475493451 container cleanup 2d75d5ec4ee0eeb2ac34136bd8db10cccef7b38d9530937752ef19964258e4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T04:12:13.887 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:13 smithi019 bash[93142]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-g 2024-09-19T04:12:14.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T04:12:14.137 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:13 smithi019 podman[93155]: 2024-09-19 04:12:13.885999351 +0000 UTC m=+0.577892184 container remove 2d75d5ec4ee0eeb2ac34136bd8db10cccef7b38d9530937752ef19964258e4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T04:12:14.138 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:14 smithi019 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.g.service: Deactivated successfully. 2024-09-19T04:12:14.138 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:14 smithi019 systemd[1]: Stopped Ceph mds.g for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:12:14.138 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 19 04:12:14 smithi019 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.g.service: Consumed 1.386s CPU time. 2024-09-19T04:12:14.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:14.149 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T04:12:14.149 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T04:12:14.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.j 2024-09-19T04:12:14.431 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:12:14 smithi019 systemd[1]: Stopping Ceph mds.j for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:14.931 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:12:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j[69997]: 2024-09-19T04:12:14.479+0000 7ffb1cc4d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:14.931 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:12:14 smithi019 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j[69997]: 2024-09-19T04:12:14.479+0000 7ffb1cc4d640 -1 mds.j *** got signal Terminated *** 2024-09-19T04:12:16.615 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:12:16 smithi019 podman[93265]: 2024-09-19 04:12:16.327154474 +0000 UTC m=+1.958723990 container died 6387b03525af81f0595d369e2d4102f27c0f5da42b3547f74a3bf196a2f06eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T04:12:16.931 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:12:16 smithi019 podman[93265]: 2024-09-19 04:12:16.788392204 +0000 UTC m=+2.419961762 container cleanup 6387b03525af81f0595d369e2d4102f27c0f5da42b3547f74a3bf196a2f06eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T04:12:16.931 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 19 04:12:16 smithi019 bash[93265]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-j 2024-09-19T04:12:17.179 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T04:12:17.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:17.212 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T04:12:17.212 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T04:12:17.212 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.b 2024-09-19T04:12:17.520 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:12:17 smithi111 systemd[1]: Stopping Ceph mds.b for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:17.905 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:12:17 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: 2024-09-19T04:12:17.518+0000 7fb216906640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:17.905 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:12:17 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b[65304]: 2024-09-19T04:12:17.518+0000 7fb216906640 -1 mds.b *** got signal Terminated *** 2024-09-19T04:12:21.569 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:12:21 smithi111 podman[69309]: 2024-09-19 04:12:21.308991374 +0000 UTC m=+3.903745474 container died f4d3c6ec631bff6d0906979b91e518f2fb99413ad352bd4933aa318866765b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T04:12:21.905 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:12:21 smithi111 podman[69309]: 2024-09-19 04:12:21.651362889 +0000 UTC m=+4.246116978 container cleanup f4d3c6ec631bff6d0906979b91e518f2fb99413ad352bd4933aa318866765b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:12:21.905 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:12:21 smithi111 bash[69309]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b 2024-09-19T04:12:21.905 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 19 04:12:21 smithi111 podman[69321]: 2024-09-19 04:12:21.810921983 +0000 UTC m=+0.498399495 container remove f4d3c6ec631bff6d0906979b91e518f2fb99413ad352bd4933aa318866765b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T04:12:22.037 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T04:12:22.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:22.071 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T04:12:22.071 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T04:12:22.071 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.e 2024-09-19T04:12:22.397 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:12:22 smithi111 systemd[1]: Stopping Ceph mds.e for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:22.655 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:12:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e[64686]: 2024-09-19T04:12:22.395+0000 7f666888c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:22.655 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:12:22 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e[64686]: 2024-09-19T04:12:22.395+0000 7f666888c640 -1 mds.e *** got signal Terminated *** 2024-09-19T04:12:25.624 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:12:25 smithi111 podman[69431]: 2024-09-19 04:12:25.315180658 +0000 UTC m=+3.030645550 container died afee5bda59ab334380dc7d6e00320c049937f4a12d82a999f88e82ba1a27c31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T04:12:25.905 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:12:25 smithi111 podman[69431]: 2024-09-19 04:12:25.730236827 +0000 UTC m=+3.445701716 container cleanup afee5bda59ab334380dc7d6e00320c049937f4a12d82a999f88e82ba1a27c31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T04:12:25.905 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:12:25 smithi111 bash[69431]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e 2024-09-19T04:12:26.209 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 19 04:12:25 smithi111 podman[69443]: 2024-09-19 04:12:25.956627033 +0000 UTC m=+0.639877418 container remove afee5bda59ab334380dc7d6e00320c049937f4a12d82a999f88e82ba1a27c31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T04:12:26.213 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T04:12:26.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:26.249 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T04:12:26.249 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T04:12:26.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.h 2024-09-19T04:12:26.602 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:12:26 smithi111 systemd[1]: Stopping Ceph mds.h for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:26.905 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:12:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h[64991]: 2024-09-19T04:12:26.600+0000 7f81bd317640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:26.905 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:12:26 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h[64991]: 2024-09-19T04:12:26.600+0000 7f81bd317640 -1 mds.h *** got signal Terminated *** 2024-09-19T04:12:29.610 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:12:29 smithi111 podman[69563]: 2024-09-19 04:12:29.303836002 +0000 UTC m=+2.820468453 container died affc28ae2aa2fefd35a37377bcf1c0843a282874ccbe9c8fb4a781e11809b9b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T04:12:29.905 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:12:29 smithi111 podman[69563]: 2024-09-19 04:12:29.692066584 +0000 UTC m=+3.208699024 container cleanup affc28ae2aa2fefd35a37377bcf1c0843a282874ccbe9c8fb4a781e11809b9b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T04:12:29.905 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 19 04:12:29 smithi111 bash[69563]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-h 2024-09-19T04:12:30.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T04:12:30.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:30.153 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T04:12:30.153 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T04:12:30.153 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.k 2024-09-19T04:12:30.480 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:12:30 smithi111 systemd[1]: Stopping Ceph mds.k for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:30.905 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:12:30 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k[64384]: 2024-09-19T04:12:30.478+0000 7ffbc08b6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:30.905 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:12:30 smithi111 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k[64384]: 2024-09-19T04:12:30.478+0000 7ffbc08b6640 -1 mds.k *** got signal Terminated *** 2024-09-19T04:12:33.589 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:12:33 smithi111 podman[69685]: 2024-09-19 04:12:33.309912671 +0000 UTC m=+2.942272451 container died 120cf7baa660cb2da2c1e0354e05d80181056525be39b54e257ef0971e735b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T04:12:33.905 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:12:33 smithi111 podman[69685]: 2024-09-19 04:12:33.687528397 +0000 UTC m=+3.319888158 container cleanup 120cf7baa660cb2da2c1e0354e05d80181056525be39b54e257ef0971e735b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T04:12:33.905 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 19 04:12:33 smithi111 bash[69685]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-k 2024-09-19T04:12:34.074 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T04:12:34.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:34.106 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T04:12:34.106 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T04:12:34.106 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.c 2024-09-19T04:12:34.458 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:12:34 smithi196 systemd[1]: Stopping Ceph mds.c for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:34.717 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:12:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c[65484]: 2024-09-19T04:12:34.466+0000 7fad524b6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:34.719 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:12:34 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c[65484]: 2024-09-19T04:12:34.466+0000 7fad524b6640 -1 mds.c *** got signal Terminated *** 2024-09-19T04:12:37.668 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:12:37 smithi196 podman[69841]: 2024-09-19 04:12:37.299201426 +0000 UTC m=+3.001966905 container died 12a2b6b5c2023b3f2df14e3fc0c89c1a06ff1c4dcc6f65b1c8525a22ce8e94ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T04:12:37.958 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:12:37 smithi196 podman[69841]: 2024-09-19 04:12:37.758575307 +0000 UTC m=+3.461340829 container cleanup 12a2b6b5c2023b3f2df14e3fc0c89c1a06ff1c4dcc6f65b1c8525a22ce8e94ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T04:12:37.958 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 04:12:37 smithi196 bash[69841]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-c 2024-09-19T04:12:38.197 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T04:12:38.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:38.232 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T04:12:38.232 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T04:12:38.232 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.f 2024-09-19T04:12:38.564 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:12:38 smithi196 systemd[1]: Stopping Ceph mds.f for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:38.958 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:12:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f[65184]: 2024-09-19T04:12:38.562+0000 7fdce9823640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:38.958 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:12:38 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f[65184]: 2024-09-19T04:12:38.562+0000 7fdce9823640 -1 mds.f *** got signal Terminated *** 2024-09-19T04:12:41.589 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:12:41 smithi196 podman[69973]: 2024-09-19 04:12:41.313190872 +0000 UTC m=+2.868875897 container died e31700601e2aa0a989806ea95969745683473d40fb0d7a4aaeb17c8d05e50e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T04:12:41.947 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:12:41 smithi196 podman[69973]: 2024-09-19 04:12:41.695586488 +0000 UTC m=+3.251271512 container cleanup e31700601e2aa0a989806ea95969745683473d40fb0d7a4aaeb17c8d05e50e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T04:12:41.947 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 04:12:41 smithi196 bash[69973]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-f 2024-09-19T04:12:42.097 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T04:12:42.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:42.132 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T04:12:42.132 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T04:12:42.132 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.i 2024-09-19T04:12:42.708 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:12:42 smithi196 systemd[1]: Stopping Ceph mds.i for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:42.708 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:12:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i[65786]: 2024-09-19T04:12:42.448+0000 7f7513234640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:42.708 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:12:42 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i[65786]: 2024-09-19T04:12:42.448+0000 7f7513234640 -1 mds.i *** got signal Terminated *** 2024-09-19T04:12:45.458 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:12:45 smithi196 podman[70093]: 2024-09-19 04:12:45.293947712 +0000 UTC m=+2.945567974 container died c1594205ad50ad907842433c69a84ddb218644c3418e87749c509c95c05dec3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T04:12:45.958 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:12:45 smithi196 podman[70093]: 2024-09-19 04:12:45.694372023 +0000 UTC m=+3.345992285 container cleanup c1594205ad50ad907842433c69a84ddb218644c3418e87749c509c95c05dec3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T04:12:45.958 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 04:12:45 smithi196 bash[70093]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-i 2024-09-19T04:12:46.099 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T04:12:46.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:46.133 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T04:12:46.133 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T04:12:46.133 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.l 2024-09-19T04:12:46.708 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:46 smithi196 systemd[1]: Stopping Ceph mds.l for 0518de98-763b-11ef-bceb-c7b262605968... 2024-09-19T04:12:46.708 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:46 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l[64882]: 2024-09-19T04:12:46.443+0000 7fa9ab93b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:12:46.708 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:46 smithi196 ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l[64882]: 2024-09-19T04:12:46.443+0000 7fa9ab93b640 -1 mds.l *** got signal Terminated *** 2024-09-19T04:12:49.708 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:49 smithi196 podman[70215]: 2024-09-19 04:12:49.302083711 +0000 UTC m=+2.953733175 container died ec465ec7efa530e2c8b8f765cc3225bfe49bbc777077b77960901776759d420a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T04:12:50.037 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:49 smithi196 podman[70215]: 2024-09-19 04:12:49.852592153 +0000 UTC m=+3.504241617 container cleanup ec465ec7efa530e2c8b8f765cc3225bfe49bbc777077b77960901776759d420a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T04:12:50.038 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:49 smithi196 bash[70215]: ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l 2024-09-19T04:12:50.274 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T04:12:50.295 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:50 smithi196 podman[70227]: 2024-09-19 04:12:50.0368881 +0000 UTC m=+0.726376062 container remove ec465ec7efa530e2c8b8f765cc3225bfe49bbc777077b77960901776759d420a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-0518de98-763b-11ef-bceb-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T04:12:50.295 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:50 smithi196 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-19T04:12:50.295 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:50 smithi196 systemd[1]: Stopped Ceph mds.l for 0518de98-763b-11ef-bceb-c7b262605968. 2024-09-19T04:12:50.296 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 04:12:50 smithi196 systemd[1]: ceph-0518de98-763b-11ef-bceb-c7b262605968@mds.l.service: Consumed 1.385s CPU time. 2024-09-19T04:12:50.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T04:12:50.306 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T04:12:50.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0518de98-763b-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:12:50.522 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T04:12:53.095 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0518de98-763b-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:12:53.295 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T04:12:55.665 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0518de98-763b-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T04:12:55.865 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T04:12:58.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:12:58.289 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:12:58.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:12:58.345 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T04:12:58.348 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi019/crash 2024-09-19T04:12:58.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:12:58.387 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:12:58.387 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:12:58.390 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi111/crash 2024-09-19T04:12:58.390 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:12:58.426 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:12:58.426 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:12:58.428 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi196/crash 2024-09-19T04:12:58.429 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash -- . 2024-09-19T04:12:58.468 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/0518de98-763b-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T04:12:58.468 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-09-19T04:12:58.469 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T04:12:58.469 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-19T04:12:58.513 INFO:tasks.cephadm:Compressing logs... 2024-09-19T04:12:58.514 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:12:58.557 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:12:58.559 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:12:58.582 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T04:12:58.583 INFO:teuthology.orchestra.run.smithi111.stderr:: No such file or directory 2024-09-19T04:12:58.583 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T04:12:58.583 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T04:12:58.583 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:12:58.583 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T04:12:58.583 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T04:12:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-19T04:12:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:: No such file or directory 2024-09-19T04:12:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T04:12:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T04:12:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.log: /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:12:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:12:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T04:12:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:12:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.a.log: 92.8%gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T04:12:58.593 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-19T04:12:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T04:12:58.593 INFO:teuthology.orchestra.run.smithi019.stderr: 82.7%gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T04:12:58.593 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:12:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.a.log: /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.2% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr: 82.7% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T04:12:58.597 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:12:58.598 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T04:12:58.598 INFO:teuthology.orchestra.run.smithi196.stderr: 90.4% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:12:58.598 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T04:12:58.598 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.l.log: 86.3% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:12:58.598 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log: 90.3% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.y.log: 92.2% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T04:12:58.599 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log: 91.0% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:12:58.600 INFO:teuthology.orchestra.run.smithi111.stderr: 86.3% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:12:58.600 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T04:12:58.600 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T04:12:58.602 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:12:58.602 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T04:12:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log 2024-09-19T04:12:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T04:12:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log: 91.1% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T04:12:58.604 INFO:teuthology.orchestra.run.smithi019.stderr: 86.3% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T04:12:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T04:12:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.d.log: --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T04:12:58.605 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.4% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T04:12:58.605 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T04:12:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.x.log: 94.7% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T04:12:58.610 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.h.log: /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T04:12:58.617 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T04:12:58.620 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 93.8% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T04:12:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T04:12:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T04:12:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.3.log: 93.9% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T04:12:58.625 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.c.log: 94.0% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T04:12:58.625 INFO:teuthology.orchestra.run.smithi019.stderr: 93.9% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T04:12:58.631 INFO:teuthology.orchestra.run.smithi111.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:12:58.635 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T04:12:58.636 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:12:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T04:12:58.652 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T04:12:58.672 INFO:teuthology.orchestra.run.smithi111.stderr: 93.6% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T04:12:58.672 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T04:12:58.707 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T04:12:58.754 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T04:12:58.793 INFO:teuthology.orchestra.run.smithi196.stderr: 91.8% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T04:12:58.854 INFO:teuthology.orchestra.run.smithi111.stderr: 91.8% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T04:12:59.046 INFO:teuthology.orchestra.run.smithi111.stderr: 95.8% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T04:12:59.199 INFO:teuthology.orchestra.run.smithi019.stderr: 90.6% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T04:12:59.916 INFO:teuthology.orchestra.run.smithi019.stderr: 92.1% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T04:12:59.973 INFO:teuthology.orchestra.run.smithi111.stderr: 91.7% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T04:13:00.225 INFO:teuthology.orchestra.run.smithi196.stderr: 91.6% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T04:13:00.255 INFO:teuthology.orchestra.run.smithi019.stderr: 91.9% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T04:13:00.288 INFO:teuthology.orchestra.run.smithi019.stderr: 92.0% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T04:13:00.655 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T04:13:00.657 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-19T04:13:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m2.086s 2024-09-19T04:13:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m7.388s 2024-09-19T04:13:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.241s 2024-09-19T04:13:00.678 INFO:teuthology.orchestra.run.smithi111.stderr: 92.2% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T04:13:00.697 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T04:13:00.981 INFO:teuthology.orchestra.run.smithi111.stderr: 92.7% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T04:13:01.102 INFO:teuthology.orchestra.run.smithi111.stderr: 92.3% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T04:13:01.205 INFO:teuthology.orchestra.run.smithi196.stderr: 92.1% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T04:13:02.263 INFO:teuthology.orchestra.run.smithi111.stderr: 93.4% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T04:13:02.264 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-09-19T04:13:02.265 INFO:teuthology.orchestra.run.smithi111.stderr:real 0m3.693s 2024-09-19T04:13:02.265 INFO:teuthology.orchestra.run.smithi111.stderr:user 0m12.486s 2024-09-19T04:13:02.265 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m0.418s 2024-09-19T04:13:02.890 INFO:teuthology.orchestra.run.smithi196.stderr: 95.5% -- replaced with /var/log/ceph/0518de98-763b-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T04:13:02.892 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-09-19T04:13:02.892 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m4.318s 2024-09-19T04:13:02.892 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m10.758s 2024-09-19T04:13:02.892 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.381s 2024-09-19T04:13:02.893 INFO:tasks.cephadm:Archiving logs... 2024-09-19T04:13:02.893 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi019/log 2024-09-19T04:13:02.894 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:13:04.020 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi111/log 2024-09-19T04:13:04.021 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:13:05.798 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi196/log 2024-09-19T04:13:05.799 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:13:07.417 INFO:tasks.cephadm:Removing cluster... 2024-09-19T04:13:07.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0518de98-763b-11ef-bceb-c7b262605968 --force 2024-09-19T04:13:07.628 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T04:13:07.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0518de98-763b-11ef-bceb-c7b262605968 --force 2024-09-19T04:13:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T04:13:08.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0518de98-763b-11ef-bceb-c7b262605968 --force 2024-09-19T04:13:08.572 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 0518de98-763b-11ef-bceb-c7b262605968 2024-09-19T04:13:08.839 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T04:13:08.839 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:13:08.859 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:13:08.875 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T04:13:08.891 INFO:tasks.cephadm:Teardown complete 2024-09-19T04:13:08.891 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T04:13:08.903 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T04:13:08.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:13:08.905 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:13:08.917 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:13:09.029 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T04:13:09.040 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T04:13:09.041 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:13:09.044 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:13:09.046 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:13:09.058 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:13:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:13:09.060 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:13:09.063 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:13:09.063 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-19T04:13:09.063 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 +126us[ +156us] +/- 35ms 2024-09-19T04:13:09.063 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -112us[ -112us] +/- 48ms 2024-09-19T04:13:09.063 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -106us[ -76us] +/- 73ms 2024-09-19T04:13:09.063 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:13:09.065 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:13:09.065 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-19T04:13:09.065 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +131us[ +153us] +/- 34ms 2024-09-19T04:13:09.065 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -123us[ -123us] +/- 48ms 2024-09-19T04:13:09.065 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -134us[ -134us] +/- 73ms 2024-09-19T04:13:09.065 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:13:09.066 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:13:09.066 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-09-19T04:13:09.066 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +125us[ +128us] +/- 35ms 2024-09-19T04:13:09.066 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -118us[ -118us] +/- 48ms 2024-09-19T04:13:09.066 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -137us[ -134us] +/- 73ms 2024-09-19T04:13:09.066 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:13:09.067 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T04:13:09.078 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T04:13:09.079 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T04:13:09.090 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:13:09.189 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:13:09.256 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T04:13:09.322 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:13:09.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:13:09.354 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:13:09.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:13:09.382 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:13:09.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:13:09.410 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T04:13:09.421 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T04:13:09.432 INFO:teuthology.task.internal:Duration was 1412.925236 seconds 2024-09-19T04:13:09.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T04:13:09.443 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T04:13:09.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:13:09.445 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:13:09.448 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:13:09.484 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:13:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:13:09.489 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:13:09.948 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T04:13:09.948 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-09-19T04:13:09.948 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:13:09.971 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-09-19T04:13:09.971 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:13:10.015 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-09-19T04:13:10.016 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:13:10.040 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T04:13:10.041 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:13:10.045 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:13:10.059 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:13:10.106 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T04:13:10.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:13:10.115 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:13:10.126 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:13:10.771 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T04:13:10.783 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T04:13:10.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:13:10.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:13:10.835 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:13:10.861 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T04:13:10.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:13:10.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:13:10.880 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:13:10.899 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-09-19T04:13:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-09-19T04:13:10.925 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-09-19T04:13:10.949 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:13:10.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:13:10.968 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:13:10.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:13:10.992 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:13:11.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:13:11.006 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T04:13:11.017 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T04:13:11.018 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi019 2024-09-19T04:13:11.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:13:11.079 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi111 2024-09-19T04:13:11.079 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:13:11.115 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911619/remote/smithi196 2024-09-19T04:13:11.115 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:13:11.151 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T04:13:11.152 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:13:11.154 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:13:11.156 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:13:11.202 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T04:13:11.215 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T04:13:11.215 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T04:13:11.230 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T04:13:11.230 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:13:11.232 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:13:11.235 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:13:11.246 INFO:teuthology.orchestra.run.smithi019.stdout: 265607 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:13 /home/ubuntu/cephtest 2024-09-19T04:13:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:13 /home/ubuntu/cephtest 2024-09-19T04:13:11.256 INFO:teuthology.orchestra.run.smithi196.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:13 /home/ubuntu/cephtest 2024-09-19T04:13:11.258 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T04:13:11.272 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T04:13:11.306 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} duration: 1412.9252355098724 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T04:13:11.307 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T04:13:11.401 INFO:teuthology.run:pass